Writing a behaviour driven API testing environment within Postman

Behaviour driven test frameworks like mocha and jasmine have certain unalienable advantages. They provide a clean way to organise tests as well as safe and (in this particular case,) isolated assertions to prevent runtime error of the testing spec itself.

For obvious reasons, having the same test specification style in Postman sounds just about right. After all, one may argue that they are testing the behaviour of the response! And not to mention the endless benefits of well organised tests, seamless creation and teardown of fixtures, yada yada yada!

the_general_problem_xkcd

I would personally love to write tests within Postman that look like the following. (Never mind the aforementioned xkcd :-/ )

Thus began my transit-time project to create a tiny (really tiny) test framework that I could include in my Postman Collections and write tests using it. I personally started using it and thought sharing it might help us BDD aficionados. The framework ended up being a gist that is about 0.7kb when minified. I named the script describe-it (such creativity!)

The “describe-it” test framework script

The following piece of minified code is the complete test framework that provides behaviour driven testing functionalities (albeit basic) similar to mocha and jasmine. The script does look cryptic — but that is to ensure that it is as lightweight as possible.

The not-so-cryptic version of the script is in the gist: https://gist.github.com/shamasis/7b12f451374314500da3 (Although, my colleagues rightfully contradict that it’s still cryptic.)

The script exposes the following functions:

  • describe (testgroup:string, tests:function);
  • it (testname:string, test:function);
  • beforeEach (setup:function);
  • afterEach (teardown:function);
  • expect (value); // assertion function

These functions work similar to mocha/jasmine and as such referring to their documentation is an obvious shortcut I would take — http://jasmine.github.io/2.0/introduction.html

The “expect” assertion function supports the following checks:

  • expect(something).toBeOk(); // allows truthy variables
  • expect(something).toNotBeOk(); // allows falsey variables
  • expect(something).toBe(somethingElse); // checks equality
  • expect(something).toNotBe(somethingElse); // checks inequality
  • expect(something).toEql(somethingElse); // checks similarity
  • expect(something).toNotEql(somethingElse); // checks dissimilarity

We might need more of these and especially ones that are related to API testing. (Would love to hear more in comments.)

Using the script within Postman

There are two parts to using this within Postman:

  1. Save the script as a global variable
  2. In the test scripts, eval the global variable before using the framework

Saving the script as a global variable

Saving the script as a global variable allows me to re-use the test framework script in multiple requests without having to manually add them. One can simply go and add the contents of the script to a global variable (or an environment variable) with name, say, describe-it-test-script. The documentation article https://www.getpostman.com/docs/environments outlines how to set and use these variables.

Personally, I save the test script as part of the first request to ensure that the test script is bundled with Postman Collections while running them in Collection Runner or Newman

The following collection fetches the test framework from the gist (so that I can keep it updated with latest content) and saves it in global variables. The subsequent requests use this global variable to get the test framework within each request’s test script.

Sample Collection that uses this test framework:
www.getpostman.com/collections/34163da95c4372e36c39

Import it in your postman and refer to the tests written in the collection called "describe-it" them.

Using the script in tests

Once you have the script stored in a variable, it is easy to use it in any test by running eval on the script. The first line of your test would look somewhat like the following:

I know — eval is evil! But that is a separate discussion altogether. If you want to avoid “eval” altogether, simply paste the minified script at the beginning of your tests. But that would make the tests look a bit ugly.

Post this, you can start using the describe, it, expect and other functions of the test framework normally.

Working with the test framework

Say, you have already saved the framework in a global variable, and you would like to run a test that ensures that a request responds with a 200 status code and serves HTML content type. The tests for the same would look like:

We can write as many tests as we want and nest as many describe blocks as we need. If there are runtime errors in one it block, the other tests would still continue to execute sequentially.

Postman Collection Run The collection shared in this post (named describe-it,) has examples of tests written using this micro framework. It has tests written with failing and passing scenarios. And upon running the collection in Postman Collection Runner, it outputs the test results grouped within their respective describe blocks.

How it works

The it function executes the test parameter function within a try-catch block. If the function executes without an error, it adds to the test description parameter (along with the description of it's parent describe blocks) to the Postman tests variable and sets it to a truthy value. In the event that an expect assertion fails or some other error occurs, the tests object key is set to a falsey value.

In the above example, if all tests pass, the tests object looks like