A Student's Guide to try

version 2.8

Copyright © 1995, 2004 Department of Computer Science Rochester Institute of Technology
All Rights Reserved

1. Introduction

The try system is a set of UNIX ® programs developed at RIT and used for electronic submission of a student's work. In some courses, it is used to submit lab work; in later courses, you will also use it to submit programming projects.

try is a very flexible system that can be tailored to suit the needs of a specific course or assignment. This document is designed to help you understand try and its messages, so you know how to respond to them. It explains how try is used in this particular course, and not everything here will apply to later courses, although the remaining courses in the first two years will most likely be very similar.

2. Why try Is Used

Using try, the instructor sets up a series of commands to accept student submissions. Instructors may also set up a series of tests to determine correctness of the submission. You will know immediately whether your program compiled and how well it worked on any public test. You can then fix the problems and resubmit your work well in advance of the deadline. Only the last files are submitted are saved so you may submit several times. There is no penalty for multiple submissions on an assignment.

Note that try is not intended to be a testing tool. Typically, a few test cases and the corresponding answers will be provided but this does not ensure that your program is correct. It's your job to exhaustively test all your work and only submit when you are confident that everything is working correctly. You should expect that your instructor will run other (hidden) tests.

A disadvantage of try is that the tools used to make the comparison between your output and the answer files are not as sophisticated as you and I. This means that your output must match exactly, for all practical purposes, or it is ``incorrect''. In particular, things like extra blank lines, missing blank lines, extra spaces at the end of a line, missing punctuation, incorrect capitalization, and misspellings can cause your program to fail a test. A later section of this guide will describe the output of try and how to use it to determine what part of your output is incorrect.

3. The try command

The basic form of the try command is:

try instructor-account assignment-code file-list

The assignment-code is a name unique to each assignment which is specified in the lab or project handout; an example is lab1-3, specifying lab number 1, activity 3. The file-list is a list of files you are submitting for this assignment. The try command to use for each assignment will be given in the assignment handout.

4. Submitting Text Files

For submissions involving only ordinary text files (e.g. data files, figures, and other things that aren't source code), try only checks to be sure that you submitted a file with the appropriate name; if so, it copies the file(s) into the instructor's account.

4.1. Example - Successful submission

The following is an example of a successful submission of an ordinary text file.

mmr000[1]% try 231-grd lab3 questions hierarchy.fig
Copying files...done

ON-TIME submission of lab3

Files being saved:
hierarchy.fig questions

lab3 has been submitted.

mmr000[2]%

try provides feedback on what it is doing as it runs. This is so you know it is working, even if the systems get a little slow. When the prompt comes back (mmr000[2]%), it means that try has finished saving the named files. Note that the contents of the files are not checked in any way; only the file names are checked.

4.2. Simple errors in submissions

There are a number of things that cause try to reject your work. The following are examples of typical problems. The solution to most of these problems is to run the try command again, but with the correct arguments.

4.2.1. Example - Wrong file name

In this example the student submitted the wrong file.

mmr000[21]% try 231-grd lab3 answers hierarchy.fig
Copying files...done

ON-TIME submission of lab3
ERROR: Missing required file: questions
ERROR: Unexpected file: answers
***** Submission unsuccessful

4.2.2. Example - Wrong assignment-code

This one is hard to see, but happens frequently to beginners. Look closely at the command (labl is NOT the same as lab1 - the last character is different: one is the letter ``el'' and the other is the number one). Those these look very similar, they are quite different.

mmr000[22]% try 231-grd labl questions

try: Instructor "231-grd" does not expect assignment "labl"!

4.2.3. Example - Wrong instructor-account

In this example, the student misspelled the instructor's account name (remember UNIX is case sensitive).

mmr000[4]% try 231-GRD lab3-1 experiment1

try: Instructor account "231-GRD" does not exist.

4.3. Example - Submitting past the due date

try will not let you submit files past the deadline for an assignment.. There is no recovery from this, as this is an error in judgement, not a simple typing error.

try 231-grd lab6-1 experiment1
Copying files...done

***** No longer accepting submissions for lab6-1
***** Submission unsuccessful

5. Submitting Source Code

When you submit source code, try, will attempt to compile your program. It may also run the program against a set of test data files established by the instructor.

For programs with textual output, your output is compared with an answer file for each test, and the results of the comparison are shown on the standard output. Some tests are ``hidden''; you are not told the results of hidden tests, nor are you told that they are even being performed.

In this course, these input test files are usually named input.x, where x is the test number. Some or all of these data files may be made available to you by your instructor.

5.1. Example - Successful submission

The following shows a successful submission of a program. The instructor defined tests for use with the program, and the user program's output for each test is stored in a file called OUT.x, where x is the test number. In this example, the user program's output matches the answer for all tests; this should be your objective for each program. After all the tests are run, try shows you what files are being saved; there may be more files saved than you submitted; this is normal.

mmr000[1]: try 241-grd lab9 Lab9.java
Copying files...done

ON-TIME submission of lab9
Compiling your program...

===== Test 1
main ticket.1 < trans.1 > OUT.1
Your output is correct!

===== Test 2
main ticket.1 < trans.2 > OUT.2
Your output is correct!

Files being saved:
Lab9.java

lab9 has been submitted.

5.2. Errors in submissions

There are many things that can go wrong when submitting a program for testing. This can cause lots of frustration, especially if you don't understand try's messages. The next sections explains some common problems and shows how to interpret try's output.

5.2.1. Example - Compile Time Errors

You are wasting your time if you don't compile and run your program before you submit it, because try won't save any files in the instructor's account if it is does not compile. Because try does quite a bit of work before it even gets to the compile step, it is much faster for you to compile it first, to be sure you have no syntax errors. Here is an example of what happens when there is a compilation error:

mmr000[1]: try 231-grd lab9 Lab9.java
Copying files...done

ON-TIME submission of lab9
Compiling your program...
Lab9.java:31: illegal start of type

for ( int i = 0; i < args.length; i++ ) {
^
Lab9.java:45: expected
}
^

lab9.java:47: 'class' or 'interface' expected
} ^ Lab9.java:48: 'class' or 'interface' expected
^
4 errors
Compilation failed.

***** No files saved.

Again, note that no files are saved, so the instructor has nothing to grade, or even to look at. The solution to this problem is to use an editor to fix the syntax errors. To be sure you fixed them all, and did not add new errors, you should compile and test the program before submitting it via try.

5.2.2. Run Time Errors

Many kinds of run time errors can occur when your program is being tested. You should test your program on your own data prior to running it via try. This is a much more time efficient and resource efficient way to debug your program. Once you are satisfied that your program works with your own tests, submit your work.

The next several subsections attempt to describe some errors that you are likely to encounter in this course. Their purpose is to show you what the output of try means. For that reason, these examples do not show the code for the programs, or how to change the code to make the output match. Also not shown are the portions of the try output prior to where testing begins, as that is not needed to analyze these types of problems.

5.2.2.1. Examples - Output doesn't match answer

The most common run time error is the failure of your output to match the correct answer. try identifies these differences to help you spot them more easily.

Example 1

(Preceding output deleted)
1 ===== Test 1
2 main ticket.1 < trans.1 > OUT.1
3
4 Your output did not match the correct output.
5 Your output:
6 Purchased 4 Grateful Dead tickets for $160.
7 Purchased 2 Elton John tickets for $70.
8 Insufficient credit for 2 Doobie Brothers tickets.
9 Insufficient credit for 1 Barbara Streisa tickets.
10 No such concert: Stones
11 Total spent $230, remaining credit: $20.
12 ----------
13 Correct output:
14 Purchased 4 Grateful Dead tickets for $160.
15 Purchased 2 Elton John tickets for $70.
16 Insufficient credit for 2 Doobie Brothers tickets.
17 Insufficient credit for 1 Barbara Streisa tickets.
18 No such concert: Stones
19
20 Total spent: $230, remaining credit: $20.
21 ----------
22 Differences (OUT.1 is yours; answer.1 is correct):
23 *** OUT.1 Mon Jul 18 14:31:06 1994
24 --- answer.1 Mon Jul 18 14:30:05 1994
25 ***************
26 *** 3,6 ****
27 Insufficient credit for 2 Doobie Brothers tickets.
28 Insufficient credit for 1 Barbara Streisa tickets.
29 No such concert: Stones
30 ! Total spent $230, remaining credit: $20.
31 --- 3,7 ----
32 Insufficient credit for 2 Doobie Brothers tickets.
33 Insufficient credit for 1 Barbara Streisa tickets.
34 No such concert: Stones
35 ! Total spent: $230, remaining credit: $20.
36
37 Files being saved:
38 concert_list.e concert_name.e concert_rec.e main.e
39
40 lab9 has been submitted WITH ERRORS.

When your program fails a test, try will print the message Your output did not match the correct output; this is shown on line 14. Then it will print all of your output, all of the correct output, and the differences between them. In the example above, the student's output appears between lines 5 and 12, the correct output appears between line 13 and 21, and the differences begin at line 22. The differences section of the output is the most useful in identifying problems.

Some try scripts use a program named diff to generate the differences section of the output; you can read the manual page on it if you want more information. diff compares one file to another, and displays sections of each file in areas where they don't match. The differences section begins with the identification of the files involved; in this example OUT.1 was the student's output, and answer.1 was the correct output; this information is given in line 22 Lines 23 and 24 show how the output from each of these files will be identitified: lines shown from the file OUT.1 will be identified with asterisks, and lines shown from the file answer.1 will be identified with dashes.

This is followed by one or more sets of differences, each beginning with a row of asterisks, as in line 25 above. The asterisks in line 26 indicates that the output below is from the file OUT.1 and that it is lines three through six from that file. Line 31 has dashes, which means that what follows are lines three through seven of the file answer.1, the correct output.

The actual lines from each file appear immediately after these heading. Lines that are identical in both files are preceded by spaces. Lines preceded by an ! exist in both files, but are somehow different. For example, the difference between lines 30 and 35 is that there is no colon after the word ``spent'' on line 30.

Example 2

(Preceding output deleted)
1 ===== Test 1
2 Sample < input.1 > OUT.1
3 3
4 Your output did not match the correct output.
5 Your output:
6 input 2 integers
7
8 the 2 numbers are: 5 5
9
10 the sum is 10
11
12 the average is: 5.00 5
13
14 ----------
15 Correct output:
16 input 2 integers
17 the 2 numbers are: 5 5
18
19 the sum is 10
20 the average is: 5.00 5
21 ----------
22 Differences( OUT.1 is yours; answer.1 is correct):
23 *** OUT.1 Mon Mar 9 13:52:33 1992
24 --- answer.1 Fri Feb 28 13:07:08 1992
25 ***************
26 *** 1,8 ****
27 input 2 integers
28 -
29 the 2 numbers are: 5 5
30
31 the sum is 10
32 -
33 the average is: 5.00 5
34 -
35 --- 1,5 ----

In the differences section, lines in your output that are preceded by a - are extra lines that are not in the correct output. Notice that no lines from answer.1 are shown at all. Line 35 indicates that lines 1 through 5 in answer.1 correspond to the lines shown above for OUT.1; the fact that they are not shown indicates that except for the extra lines noted above, they are identical to the corresponding lines from OUT.1. In this example, the student double spaced all the output, which is why it didn't match the answer.

Also, lines in the correct answer section which are preceded by a + are lines which are missing in your output. The example above does not illustrate this.

The next example shows several different areas in the student's output that didn't match the answer.

Example 3

(Preceding output deleted)
1 ===== Test 1
2 Project3 < input3.1 > OUT3.1
3 Your output is correct!
4
5 ===== Test 2
6 Project3 < input3.2 > OUT3.2
7
8 Your output did not match the correct output.
9 Your output:
10 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit): item number? quantity? item number? quantity? item number? Does New York tax apply? (y or n) Does California tax apply? (y or n)
11 Number of items ordered: 3
12 Your subtotal is: $41.97
13 Your tax is: $.00
14 Your total is: $41.97
15
16 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit):
17
18 Item number 0; number ordered 0
19 Item number 1; number ordered 1
20 Item number 2; number ordered 2
21 Item number 3; number ordered 0
22 Item number 4; number ordered 1
23 Item number 5; number ordered 0
24 Item number 6; number ordered 0
25 Item number 7; number ordered 0
26 Item number 8; number ordered 0
27 Item number 9; number ordered 0
28
29 Summary of orders
30
31 number of customers 1
32 total amount to collect 41.97
33 tax due New York State .00
34 tax due California .00
35 our income 41.97
36
37 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit):
38 ----------
39 Correct output:
40 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit): item number? quantity? item number? quantity? item number? Does New York tax apply? (y or n) Does California tax apply? (y or n)
41 Number of items ordered: 3
42 Your subtotal is: $41.97
43 Your tax is: $.00
44 Your total is $41.97
45
46 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit):
47
48 Item number 0; number ordered 0
49 Item number 1; number ordered 1
50 Item number 2; number ordered 2
51 Item number 3; number ordered 0
52 Item number 4; number ordered 0
53 Item number 5; number ordered 0
54 Item number 6; number ordered 0
55 Item number 7; number ordered 0
56 Item number 8; number ordered 0
57 Item number 9; number ordered 0
58
59 Summary of orders
60
61 number of customers 1
62 total amount to collect 41.97
63 tax due New York State .00
64 tax due California .00
65 our income 41.97
66
67 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit):
68 ----------
69 Differences (OUT3.2 is yours; answer3.2 is correct):
70 *** OUT3.2 Mon Mar 9 13:46:43 1992
71 --- answer3.2 Thu Feb 6 08:50:19 1992
72 ***************
73 *** 2,8 ****
74 Number of items ordered: 3
75 Your subtotal is: $41.97
76 Your tax is: $.00
77 ! Your total is: $41.97
78
79 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit):
80
81 --- 2,8 ----
82 Number of items ordered: 3
83 Your subtotal is: $41.97
84 Your tax is: $.00
85 ! Your total is $41.97
86
87 enter menu selection n(ew order), l(ong display), s(hort display), ^D(exit):
88
89 ***************
90 *** 10,16 ****
91 Item number 1; number ordered 1
92 Item number 2; number ordered 2
93 Item number 3; number ordered 0
94 ! Item number 4; number ordered 1
95 Item number 5; number ordered 0
96 Item number 6; number ordered 0
97 Item number 7; number ordered 0
98 --- 10,16 ----
99 Item number 1; number ordered 1
100 Item number 2; number ordered 2
101 Item number 3; number ordered 0
102 ! Item number 4; number ordered 0
103 Item number 5; number ordered 0
104 Item number 6; number ordered 0
105 Item number 7; number ordered 0

This program passed the first test (shown in line 3), but not the second; it is not unusual to have a mix of successful and unsuccessful tests. In test two, two sets of differences are shown. The first appears in lines 73 through 88 above, and the second in lines 90 through 105. Line 73 indicates that the first problem occurred somewhere in lines 2-8 in the OUT3.2 file; line 77 identifies the specific line that doesn't match. Line 81 indicates that lines 2-8 of answer3.2 are shown, and line 85 identifies the line that doesn't match. Line 90 indicates that the next problem occurred somewhere in lines 10-16 of OUT3.2, and line 94 identifies the particular line in question. Output lines that are not shown as differences are correct.

5.2.2.1.1. Seeing differences in very long outputs

If the output of try is too long to fit in the window, you may wish to use the scrollbars on your shell window (if it has them) so that you can view different parts of the output by moving back and forth. Another way to accomplish the same result is to save the output in a file and then examine it with an editor. This is most easily done by saving the standard output when try is executed, as the following example does:

mmr0000[24]: try 231-grd lab4-1 | tee output


5.2.2.1.2. Seeing differences in very wide outputs

If the output lines are too wide for the window, some types of windows cannot show the remaining characters. This can present problems if your output error is beyond the edge of the window. If you see no error in a line identified as being different than the answer, first try expanding the width of the window. If that still doesn't show the entire line, you can either rerun try in a different kind of window (i.e. an xterm, xvt, or shelltool), which will wrap the extra characters onto additional lines. Or you can capture the try output as described above, and then peruse the output file with an editor.

5.2.2.2. Example - Missing newlines

Another common problem is to forget to do an io.new_line after a line of output. This statement writes the character that marks the end of an output line. If it is omitted, your output will either be missing a blank line, or two lines in your output will be concatenated, that is, merged into one.

But the last line in the file should also be terminated with a newline character, and if that is missing you will get output like this:

(Preceding output deleted)
1 ===== Test 1
2 Sample < input.1 > OUT.1
3
4 Your output did not match the correct output.
5 It is missing a newline at the end of the last line, but is otherwise ok.
6
7 ===== Test 2
8 Sample < input.2 > OUT.2
9
10 Your output did not match the correct output.
11 Your output (which had no newline on the last line):
12 input 2 integersthe 2 numbers are: 5 5
13 the sum is 10
14 the average is: 5.00 5
15 ----------
16 Correct output:
17 input 2 integers
18 the 2 numbers are: 5 5
19
20 the sum is 10
21 the average is: 5.00 5
22 ----------
23 Differences (OUT.2 is yours; answer.2 is correct):
24 *** OUT.2 Fri Jan 27 09:13:25 1995
25 --- answer.2 Fri Jan 27 09:04:45 1995
26 ***************
27 *** 1,3 ****
28 ! input 2 integersthe 2 numbers are: 5 5
29 the sum is 10
30 ! the average is: 5.00 5
31 --- 1,5 ----
32 ! input 2 integers
33 ! the 2 numbers are: 5 5
34 !
35 the sum is 10
36 ! the average is: 5.00 5

Lines 4 and 5 indicate that the output for the first test was correct except that it had no newline character at the end of the last line. Note that while there is not much wrong with this output, it is still incorrect.

The second test, though, is worse. Line 11 indicates that it too had no newline after the last line. It is also missing the newline at the end of the first line of output, which is why the first two lines are concatenated in line 12. It is also missing the second newline after what should be the second line of output, which is why the empty line shown in the correct output (line 19) does not appear. The differences section points out both of these errors.

The difference section also flags two lines (30 and 36) which appear to be identical. In fact they are different: remember it said earlier that the student's output had no newline at the end of the last line. The correct output does, and that is the difference being indicated here. The solution is simple, add an io.new_line statement after the last line of output.

Note that if the correct output has no newline after its last line, then your output should not either. This situation is rare, though, and will be clearly stated in the program specifications if it ever occurs.

5.2.2.3. Standard output versus standard error

Some programs are supposed to print things to the standard error. try tests these programs by separating the standard error from the standard output and saving them individually. Each is then tested with its own answer files. The way in which the standard error output is tested is exactly the same as for the standard output. The messages printed by try identify this as ``standard error output'' to distinguish it from ordinary output.

5.2.2.4. Retesting single failed tests

For some assignment there are many tests run on your program. If your program fails a small number of tests, you may not want to run all the tests to see if your modified program is correct. You can run your program with the appropriate datafile, which should be available in the icss231 account, and send the output to a file. Run the demonstration program we provide with the same data file and send the output to a second file. Then run

diff -cw 0your-output-file demo-output-file

6. What try saves

By now you should have noticed that try saves certain files for you. If you send it the correct arguments, and for program submissions your program compiles, and if you let try execute to completion, then it will save files for you in the instructor's account. If you interrupt try (via ctrl-c), or provide the wrong files, or your program won't compile, then nothing is saved from the current submission attempt, the instructor has nothing to grade, and you will have nothing to show for your effort.

try will tell you what files it is saving. Which specific files are saved varies somewhat depending on the particular assignment, and how well your program performed. Minimally, files in the file-list on the command line are saved.

You should be aware that try only keeps the files created by the most recent submission of each assignment, and all files from previous submissions are deleted when the new files are saved. Also, try keeps a log for each attempt indicating whether or not it was successful. This last feature means the instructor can tell if you ever attempted to submit a particular assignment, and some general information about whether it succeeded or not.

6.1. Finding out what try has saved for you.

Sometimes you can't remember whether you submitted a lab experiment or a project, or if there was anything wrong with it. There is a capability to query try to tell you what it knows about your assignment's status. The command is:

try -q instructor-account assignment-code

7. Things that require an Instructor's intervention

Since try is a complex system to set up, occasionally errors occur which require that an instructor get involved. The following are examples of things that can only be fixed by an instructor:

try: Instructor "231-grd" does not expect assignment "lab2-2"!
&../lab2/run_safe: input.1: Permission denied
&../lab2/run_safe: input.2: No such file or directory
Your account (abc1234) is not registered.

First check to make sure you spelled everything correctly, etc. If you are sure you used the correct command, then copy the exact command line and entire message returned from try and send it to your lab instructor. These are things that the lab assistants cannot fix for you, and you must see an instructor associated with the course.

There is another error that may occur, especially near midnight on the night assignments are due, and that is ``file system full'' (or some similar message). Since lots of files are created temporarily while you submit an assignment, it is very likely that this problem will disappear shortly; try again in a minute or so. If because of one of these problems it gets to be past the deadline, send a copy of the exact message to your lab instructor and ask what to do about it. DO NOT modify any of your files after the deadline, or you won't be able to prove that you had the work done within the time limit. It is your instructor's right to refuse to accept the assignment if it is late, regardless of the reason.

8. try and Due Dates

try automatically checks due dates when you submit either labs or projects. For labs, there is only one period of time that is significant: the ``window'' is any time prior to the deadline.

For courses with projects, there may be two periods that are significant: the ``on-time window'' is any time prior to the initial deadline, and the ``late window'' is any time after the initial deadline but before the final deadline for a project. Not all projects will have a late period; the project handout will clearly state the presence or absence of a late period. try allows projects to be submitted during the on-time window and during the late window. Projects submitted during the late window are recorded as being late, and penalties will be applied as outlined in the syllabus. Only the last version submitted is graded. After the late window, try refuses to accept projects.