Pardon Collections

This tutorial builds on the concepts explained in Pardon Templates,

Pardon Templates A quick introduction to Pardon's templates

Services and Endpoints

A Pardon collection is generally one or more services anchored by a service.yaml file and with each endpoint represented by an https (HTTP Schema) file.

Schema files are a sequence of http request and response templates, and optionally some additional inline configuration.

A basic collection defining a service and the actions for a products REST API might look like this.

Directorycollection
- Directoryexample the example service
  service=“example”
  - service.yaml common configuration for all the endpoints
  - ping.https a basic endpoint
    endpoint=example/ping
    action=ping
  - Directoryproducts we can group related endpoints
    create.https create a product
    action=“create”
    endpoint=“example/products/create”
    get.https get a product
    action=“get”
    endpoint=“example/products/get”
    update.https update a product
    delete.https delete a product
    list.https list products

When developing a collection, one might start with a service.yaml and confirm the setup with something like a ping.https endpoint.

service.yaml
ping.https

The service.yaml file contains common specifications like config, defaults, and mixins (which we’ll get to later in this tutorial).

config:
  origin:
    env:
      prod: https://example.com
      stage: https://stage.example.com

And our ping.https file specifies how to make the ping call to the service.

>>>
GET https://example.com/ping

So far, so good: we should be able to build requests for both stage and production origins.

Input values

The input request is the “most direct” mechanism for getting data into Pardon, but not everything can be solved without another layer of indirection!

Values can supply data to templates, and can even be used to rewrite templates where config alternatives are involved.

In these exercises a new input field for values is available to explore this mechanism (the input).

Exercises

Values input
Values override

Try setting a value in the values input.

config choices can be overridden with values. This allows any request to be easily repurposed to a different environment (in particular, replaying tests).

env=stage

Override the the request input

Set env=prod to override https://stage.example.com.

(try the other way as well).

env=prod

GET https://stage.example.com/ping

Loading Pardon Playground...

When using the pardon commandline utility, values can be specified with key=value syntax anywhere in the arguments.

% pardon https://example.com/ping env=stage --http
curl "https://stage.example.com/ping"

We can also omit any http or URL data and select an endpoint directly

% pardon endpoint=example/ping env=stage --http
GET https://stage.example.com/ping

A RESTful resource

The products lifecycle would also defined with https files:

For product creation, this template only defines the name field (which makes it mandatory),

>>>
POST https://example.com/products
Content-Type: application/json

{
  "name": "{{name}}"
}

Defining the response template instructs pardon to extract the product value from the response, assuming it matches. Equally importantly, it provides a little documentation about the shape of the response for humans. We will cover dataflow in a later section, as it’s more of an advanced topic for scripting.

The REST of the product resources endpoints are specified minimally here, just enough to distinguish them from each other.

Requests info for a single product.

>>>
GET https://example.com/products/{{product}}

Updates info for a single product.

>>>
PUT https://example.com/products/{{product}}
Content-Type: application/json

{}

Deletes a single product.

>>>
DELETE https://example.com/products/{{product}}

Lists products.

>>>
GET https://example.com/products

With all this set up we can finally explore how pardon works with a collection of requests.

Exercises

Set a value via values.

Notice how we can provide the {{name}} value here without having to specify a request body at all!

Many simple requests can be parameterized to take advantage of this abbreviated input mechanism.

env=stage name=thneed

Add a parameterized query param

In the spirit of progressive enhancement, we can add a parameterized value in our input,

GET https://stage.example.com/products?type={{type}}

and then define it.

env=prod type=pencils

How can we make supplying the type optional?

Select the endpoint directly with parameterization.

The service, action, and endpoint values can be used to select (or at least dramatically narrow down) which endpoint templates Pardon even attempts to evaluate.

First, please delete the entire input. We don’t need it!

POST https://example.com/products

Then select the endpoint directly.

endpoint=example/products/list

you can try different endpoints, and/or env=stage if you like, for the get/update/delete endpoints, you’ll need to specify a product=... value as well.

This only works for GET method endpoints.

Non-GET endpoints need the method specified as well, this adds a little friction to prevent accidental POST requests.

endpoint=example/products/create method=POST name=thneed

Loading Pardon Playground...

Mixins

Oh no. In our rush to add features, we forgot to include authorization headers to call these endpoints: the security team took care of this in the gateway on stage, and now all of our endpoints need to be updated, and tested.

We want to tell Pardon that this service needs authentication, rather than updating every endpoint. We just need to specify a mixin at the service level.

config:
  origin:
    env:
      prod: https://example.com
      stage: https://stage.example.com
mixin:
  - ./auth.mix.https

We’re putting it next to our service in the file tree. But any relative path inside the service directory is fine.

Directorycollection
- Directoryexample
  - service.yaml
  - auth.mix.https
  - ping.https
  - Directoryproducts/
    create.https
    …

The contents look (kind of) like any other https endpoint.

>>>
ANY //
Authorization: {{ @auth = `${env}-auth-token` }}

The special request method ANY matches all request methods, and the special value // matches any URL. (A “mixin” is mixed-in only if it is “compatible” with the request.)

This adds an authorization header to all requests.

HTTP requests unfortunately contain multiple types of information: information that selects an endpoint, information that we want to send to the endpoint, and information that we need to send to tell the server who we are. This last category of values are often both security sensitve and (relatively) useless for human comprehension, so we tend to redact them.

Template values decorated with @ are considered secrets, and Pardon knows to keep them out of durable locations, and just generally hide them by default.

Commonly, we would compute the secret using an imported function.

Authorization: {{ @auth = makeAccessToken(env) }}

where makeAccessToken is an async function exported from a javascript (or typescript) file, and it could be implemented by calling another pardon endpoint.

Let’s try it out in stage and confirm all our endpoints are working and have authorization applied… (The @ in @auth marks it a secret. You can see the value using the lock button in this exercise.).

Loading Pardon Playground...

Mixin-Match

TL;DR: mixins progressively enhance the input request, but only if they match.

They do not affect whether the endpoint template matches the input request.

Awesome! We’ve got a security measure applied to our requests. And we provided confirmation of all our requests working on stage, so ops rolled these changes out to production.

Unfortunately, now alerts are firing because the (oops, only in production!) liveness probe hitting /ping requires an auth token and is not provisioned to use one. We missed this case in staging because our updated collection sent auth tokens to every endpoint.

Well, it’s not actually an outage, and the fix isn’t even on our end (ops is in control of removing the Authorization header check from the /ping endpoint), but we do want to adjust our collection to align with the deployment.

One option is removing the mixin from service.yaml and including it selectively from every authenticated endpoint.

config:
  origin:
    env:
      prod: https://example.com
      stage: https://stage.example.com
mixin:
  - ./auth.mix.https

mixin:
  - ../auth.mix.https
>>>
POST https://example.com/products

mixin:
  - ../auth.mix.https
>>>
GET https://example.com/products/{{product}}

mixin:
  - ../auth.mix.https
>>>
PUT https://example.com/products/{{product}}
Content-Type: application/json

{}

mixin:
  - ../auth.mix.https
>>>
DELETE https://example.com/products/{{product}}

mixin:
  - ../auth.mix.https
>>>
GET https://example.com/products

but that touches many files. As the collection grows, this kind of maintenance would grow as well.

It is cleaner (in this case, at least) to specify an opt-out for the few exceptional endpoints that don’t need auth, rather than opting-in. To have the mixin supply an opt-out behavior, we can add a config section.

config:
  authorization: token
>>>
ANY //
Authorization: {{ @auth = `${env}-auth-token` }}

Just like origin in the main request, this authorization value needs to match (or be not specified) in the input for this mixin to be applicable.

We can then default authorization as none in our ping endpoint (choosing any value other than token), which disables the mixin selectively.

defaults:
  authorization: none
>>>
GET https://example.com/ping

This implements the same change in behavior with two changes instead of N+1, and provides some additional controls we explore here:

Exercises

Set the endpoint to ping.

Notice the authorization header disappears as expected.

GET https://example.com/ping

Send authorization to ping anyway?

Override the default authorization to force an authenticated ping request!

authorization=token

Send an unauthenticated request to products.

Override the default authorization to force an unauthenticated products request. Why would we do this? I don’t know, maybe for testing? To assert dominance over our framework?

authorization=none

GET https://example.com/products

Loading Pardon Playground...

Collection Layering

To match team structures, collections are defined in layers:

A base layer might define the endpoints, but credentials can be defined in another. Different teams have different access to credentials to the same services, so Pardon supports separation of concerns and access requirements by allowing layers of functionality to be applied.

A root pardon config defines the layers of collections to use. We’ll need to revisit this in the section on scripting.

Next Steps

Next we will cover more dataflow mechanisms.

Pardon Dataflow How data flows through templates.

Afterwards we will revisit layering a bit in scripting.