Variations based on multiple examples for requestBody

[thim81]

We have gotten the input from several Portman users, where they would like to leverage their multiple examples used in OpenAPI in a clever way when generating Postman tests. (like #279).

Consider an OpenAPI specification with examples:

  '/mobile/v1/messages/send':
    post:
      operationId: post-mobile-send
      summary: Send mobile messages
      description: 'Send messages in bulk to a range of devices.'
      requestBody:
        content:
          application/json:
            schema:
              description: The payload for sending a messages in bulk.
              type: array
              items:
                $ref: '#/components/schemas/SendMessageBulkModel'
                examples:
                  sample-send-push-apple:
                    $ref: '#/components/examples/sample-send-push-apple'
                  sample-send-push-google:
                    $ref: '#/components/examples/sample-send-push-google'
                  sample-send-push-huawei:
                    $ref: '#/components/examples/sample-send-push-huawei'

Currently this OpenAPI gets converted into 1 Postman request, where the 1st example will be used in the Postman request body.

Ideas/feedback wanted:

• What if this could result in additional variations request variations?
  In this case, the normal Postman request would take example 1 and Portman could generate an additional Postman requests for the 2nd/3rd example as a "variation".
• How should the Portman config allow you to define this variation? Any ideas/suggestions?

{
 "variationTests": [
      {
        "openApiOperation": "*::/mobile/*",
        "openApiResponse": "200",
        "examples": ["sample-send-push-google", "sample-send-push-huawei"]
        "variations": [
          {
            "name": "Send messages",
            "xyz": "..."
          }
       ]
    }
  ]
}

I havent investigated the technical feasibility, but I'm looking for input if it is a valuable feature and how to handle the configuration with the least complexity.

[thim81]

Tagging: @AliKhalili @savage-alex @george4joseph @nicklloyd @BenjaminDC for your feedback or input on this matter?

[erwin-cuppens-personal]

@thim81 one of the challenges we have is not only that we have multiple different request types, but they also have different responses.
To be more concrete, we have an endpoint that is used to create a placeholder for html content. We have different types (e.g. html, image, css, script, ...) that all have some shared properties and some specific properties. Whenever we do a creation call, the request contains one of the specific types and the expected response should also contain the result for that specific type.
So when thinking about a Portman solution it would be good to be able to link the request to a specific response (e.g. by using the example names)

If you need any more input or examples, you know where to find me!

[savage-alex]

If this propsal for requestBody examples or also for requestResponse example checks?

We often use OpenAPI defined examples for error responses - there might be 5 different 400 response examples)

As these are a set of values vs a schema, is it possible to check the response values that they are compliant.

The variation overrides can set the scenario up and we are then checking that the schema and values are compliant vs the API definition?

Perhaps I have missed the point

[thim81]

I think we can already split up this idea in 2 potential features:

1. RequestBody variations based on the examples (which was the initial) proposal.
2. Response checks based on the examples

For this topic, lets focus first on the first feature.

I have created a seperate topic for the response validation based on an example.