Writing a behaviour driven API testing environment within Postman

Behaviour driven test frameworks like mocha and jasmine have certain unalienable advantages. They provide a clean way to organise tests as well as safe and (in this particular case,) isolated assertions to prevent runtime error of the testing spec itself.

For obvious reasons, having the same test specification style in Postman sounds just about right. After all, one may argue that they are testing the behaviour of the response! And not to mention the endless benefits of well organised tests, seamless creation and teardown of fixtures, yada yada yada!

the_general_problem_xkcd

I would personally love to write tests within Postman that look like the following. (Never mind the aforementioned xkcd :-/ )

Thus began my transit-time project to create a tiny (really tiny) test framework that I could include in my Postman Collections and write tests using it. I personally started using it and thought sharing it might help us BDD aficionados. The framework ended up being a gist that is about 0.7kb when minified. I named the script describe-it (such creativity!)

The “describe-it” test framework script

The following piece of minified code is the complete test framework that provides behaviour driven testing functionalities (albeit basic) similar to mocha and jasmine. The script does look cryptic — but that is to ensure that it is as lightweight as possible.

The not-so-cryptic version of the script is in the gist: https://gist.github.com/shamasis/7b12f451374314500da3 (Although, my colleagues rightfully contradict that it’s still cryptic.)

The script exposes the following functions:

  • describe (testgroup:string, tests:function);
  • it (testname:string, test:function);
  • beforeEach (setup:function);
  • afterEach (teardown:function);
  • expect (value); // assertion function

These functions work similar to mocha/jasmine and as such referring to their documentation is an obvious shortcut I would take — http://jasmine.github.io/2.0/introduction.html

The “expect” assertion function supports the following checks:

  • expect(something).toBeOk(); // allows truthy variables
  • expect(something).toNotBeOk(); // allows falsey variables
  • expect(something).toBe(somethingElse); // checks equality
  • expect(something).toNotBe(somethingElse); // checks inequality
  • expect(something).toEql(somethingElse); // checks similarity
  • expect(something).toNotEql(somethingElse); // checks dissimilarity

We might need more of these and especially ones that are related to API testing. (Would love to hear more in comments.)

Using the script within Postman

There are two parts to using this within Postman:

  1. Save the script as a global variable
  2. In the test scripts, eval the global variable before using the framework

Saving the script as a global variable

Saving the script as a global variable allows me to re-use the test framework script in multiple requests without having to manually add them. One can simply go and add the contents of the script to a global variable (or an environment variable) with name, say, describe-it-test-script. The documentation article https://www.getpostman.com/docs/environments outlines how to set and use these variables.

Personally, I save the test script as part of the first request to ensure that the test script is bundled with Postman Collections while running them in Collection Runner or Newman

The following collection fetches the test framework from the gist (so that I can keep it updated with latest content) and saves it in global variables. The subsequent requests use this global variable to get the test framework within each request’s test script.

Sample Collection that uses this test framework:
www.getpostman.com/collections/34163da95c4372e36c39

Import it in your postman and refer to the tests written in the collection called "describe-it" them.

Using the script in tests

Once you have the script stored in a variable, it is easy to use it in any test by running eval on the script. The first line of your test would look somewhat like the following:

I know — eval is evil! But that is a separate discussion altogether. If you want to avoid “eval” altogether, simply paste the minified script at the beginning of your tests. But that would make the tests look a bit ugly.

Post this, you can start using the describe, it, expect and other functions of the test framework normally.

Working with the test framework

Say, you have already saved the framework in a global variable, and you would like to run a test that ensures that a request responds with a 200 status code and serves HTML content type. The tests for the same would look like:

We can write as many tests as we want and nest as many describe blocks as we need. If there are runtime errors in one it block, the other tests would still continue to execute sequentially.

Postman Collection Run The collection shared in this post (named describe-it,) has examples of tests written using this micro framework. It has tests written with failing and passing scenarios. And upon running the collection in Postman Collection Runner, it outputs the test results grouped within their respective describe blocks.

How it works

The it function executes the test parameter function within a try-catch block. If the function executes without an error, it adds to the test description parameter (along with the description of it's parent describe blocks) to the Postman tests variable and sets it to a truthy value. In the event that an expect assertion fails or some other error occurs, the tests object key is set to a falsey value.

In the above example, if all tests pass, the tests object looks like

15 thoughts on “Writing a behaviour driven API testing environment within Postman”

  1. Hello Shamasis, thanks for that!
    This is a great addition to testing in Postman. Would be great to see it comes with standard Postman. Another idea is to have it as an extension or plugin.

      1. Hello @shamasis:disqus , it's mainly because the test and the results are easy to read, this turns out also to be easy to design.

  2. Itest I have done this before. I have even uses "Tests" in postman to compare between a legacy system and the new system that we are replacing said legacy system with (only works if systems are written to in parity).

    The simplest way to answer your question is via the following:

    // store the responseBody for future comparisons
    var oldResponse = postman.getEnvironmentVariable("unique-to-this-test");
    if (oldResponse === null) {
    tests['no previous value to compare'] = true;
    postman.setEnvironmentVariable("unique-to-this-test", responseBody)
    } else {
    // compare old response with current response
    // depending on what you are doing something like the following will be good enough, but this requests the responseBodies to be exact.
    tests['response matches previous'] = (oldResponse === responseBody)
    }

    If you want to do say a deepEquals() where the attributes in the responseBody are the same, but are in a different order then you will need to customize the above a bit to your specific needs. And obviously I prefer to use the behaviour testing that is mentioned in this blog post, so feel free to update the script I gave to be except(responseBody).toEql(oldResponse) instead.

    This way allows you to test any type of response and not just JSON, if you need to parse the json and compare it you should then do something different.

  3. This really need to be integrated into Postman. Atomic test is a great protection but it does not help to test complete scenarii. Kudo on this awesome premise.

  4. Is there a way to get this method to work within newman?
    When I try to load an exported environment file with the "describe-it-test-script", newman won't load it and make the test suite functions available to my test collection.

  5. Hi there,
    Any news about some ability to import and use multiple scripts in test collections.
    Tried and used the setGlobalVariable / eval trick. It works like a charm (I extensively use postman-bdd that way), but is quite limitative when having a bunch of scripts.
    One of the tighter limit with postman_collection is about source control.
    A new easy mechanism to have external libraries made available would be great for that. So would be great to not update collection and tests guids on each collection import (same as above, makes version control quite tedious).
    Anyway, thanks for your great job.

Leave a Reply

Your email address will not be published. Required fields are marked *