Command Unit Testing by Xavid


Suppose you've put great effort into making sure that a given series of commands responds a certain way. You're now ready to work on the rest of your game, but you want to make sure that nothing you do breaks what you just set up. You can have "test" commands to check this, but you have to manually verify that the output is correct each time. This extension provides another option: the ability to create 'unit tests' that run a series of commands and make assertions about the output of some of them.

Chapter 1 - Writing Tests

Unit tests are very similar conceptually to the Inform built-in "test" command, in that they run a series of user commands, but they're defined differently, in unit test rules. A basic example would be:

Unit test:
     start test "dropping";
     do "take fish";
     assert that "drop fish" produces "Dropped.";
     assert that "[the fish]" substitutes to "the dirty fish".

The "start test" instruction just prints a message to make it easier to tell where one test ends and another begins. There's also a "start suite" which works in the same way. (Because tests need to set up a series of commands and later execute them, normal "say" won't work well, but you can use "echo" to schedule other messages in the output.)

The "do" instruction just executes a command without doing anything special with it.

The "assert that ... produces" instruction is the real powerhouse; it runs a command, capturing the output text, and then compares it to the expected output. If they don't match, the extension reports an assertion failure. Ordinary newlines are removed from the end of the output.

You can also do "assert that ... substitutes to ..." to test the value an expansion produces without running any particular command.

Chapter 2 - Running Tests

You can run tests manually with the "unit" command. You could also run them automatically (on startup, say) with "try running the unit tests", but given that this extension currently doesn't support restoring the original game state this is probably not recommended.

Chapter 3 - Caveats

Section 1 - Unit Test Commands Are Delayed

The way unit tests work, we queue up a bunch of commands during the unit test rules and then run them afterwards. Thus, most non-unit-test specific things you might want to do in a unit test rule won't work, because the commands will happen too late. For example, if you do something like:

Unit test:
     do "x apple";
     now the player holds the apple;
     do "x apple".

The now line will take effect before either of the examine commands.

Section 2 - Clarification Questions

Inform doesn't give Inform 7 code easy full control over clarification questions. Assertions work properly when a clarification question is passed, but the unit testing will pause there. To let the test continue, you'll be asked to type the "z" ("wait") command to get out of the "waiting for clarification question answer" state.

(We could potentially improve this by basing the extension off of the Inform 6 implementation of the "test" command instead of a "reading a command" rule, which would let us also script answers to clarification questions, but that'd make the extension much more complicated/brittle for a pretty minor benefit.)

Section 3 - Text Capture

Assertion instructions rely on text capture (based on Text Capture by Eric Eve). On the Z-machine, if the underlying action also uses text capture (for example, it involves Implicit Actions by Eric Eve), then this won't work properly. (On Glulx, we add a second text capture implementation that can co-exist with the normal one.)

Text capture has a 256-character limit on the length of captured text; for commands with longer expected output, you might want to increase this with something like:

Use maximum unit-capture buffer length of at least 512.

or, on the Z-machine:

Use maximum capture buffer length of at least 512.

Chapter 4 - Bugs and Comments

This extension is hosted in Github at https://github.com/i7/extensions/tree/master/Xavid. Feel free to email me at extensions@xavid.us with questions, comments, bug reports, suggestions, or improvements.

Example: * Unit 1 - A basic unit test.

"Unit 1"

Include Command Unit Testing by Xavid.

Facility is a room.

A thing called an apple is here.

Unit test:
     start test "taking a possession";
     do "take apple";
     assert that "take apple" produces "You already have that.";
     assert that "[a apple]" substitutes to "an apple".

Test me with "unit".