Reproducing Issues and Testcases

One piece of “low-hanging fruit” for developers not well-versed in the Rust and C (and C++) codebase is reproducing and writing test cases for reported issues. It would be helpful to develop a way to track issues for which test cases need to be written, either as a GitHub issue label, or an issue with a checklist.

It looks GitHub #31 is a good candidate. @pkgw, are there others that could use regression tests?

@sasha Let’s see … right now GitHub #78 is the only one I see that could definitely use a test case.

The easiest sort of test case to add is one where you can input a .tex file, process it using the basic plain TeX format, and check an output .xdv file to see if the result is correct. This genre of test case should get handled in the tex-outputs testbed. The steps to add a new test are:

  1. Identify a bug that fits this description!
  2. Come up with a name for the test case that will test the solution, using underscores to separate words.
  3. Create input files in tests/tex-outputs/ using the test case name as a stem; i.e., if your test case is called the_letter_a, the files should be named like tests/tex-outputs/the_letter_a.tex.
  4. Add an entry at the bottom of tests/ that invokes the test case.
  5. At this point, you can’t make progress until the bug gets fixed!
  6. Once the bug is fixed, run Tectonic manually with the --outfmt=xdv argument to create the output .log and .xdv files in the test directory, and add them to the repository.

@pkgw Do you suggest tests/tex-ouputs/ for #31 also, or would something in the new tests/executables be appropriate?

@sasha Yeah, executables would be more appropriate for that one.

(I have to admit that, despite my interest in bootstrapping these forums, this aspect of this particular discussion is probably best carried out on GitHub :slight_smile:)

By the way, one idea I had: it might be helpful to try to turn the problems from The TeXBook into test cases. … Although they span many different styles, so lots of different little test rigs would be needed and some of them would probably just not be worth it to implement. Still, it’d be kind of fun!