A/B testing

Let's say you want to try multiple variants of a page or a feature by dividing users into different groups. All users in one group will be served a specific variant from the backend which is different for users in other groups. You can achieve this in WebSonic and also cache these multiple variants on the CDN.

The users should stick to the same group they have been assigned once. Otherwise. they will be seeing a different variant on each visit and it won't serve the purpose of A/B testing.

Understanding with an example

Let's say we want to serve two variants of a page at https://www.example.com/ab-testing-page

We will be using cookies to divide users into different groups. Here is what should happen.

A new user access the page

A new visitor accesses https://www.example.com/ab-testing-page and doesn't have a cookie with a name ab-experiment. WebSonic is configured to not cache requests which don't have this cookie. Your backend receives the request, and randomly assign this user to a group, and set the cookie value e.g. ab-experiment=version1. You can set the value of this cookie to anything e.g. version1 or version2 or version3. Now, this cookie will be sent along with all future requests based on Expiry and path set by the backend. Note that we will have to configure Websonic to not cache this request.

The first request which doesn't have any cookie should never be cached. It allows the backend to randomly assign a new user to a group and set a cookie for all future requests for that user.

This is how your configuration should look like. We have created a rule which will matches requests to the path that begins with /ab-testing/page and does not have a cookie with a name ab-experiment.

A user access the page first time will match this first rule

Subsequent request cases by an old user

Now an old user access the page with a cookie ab-experiment=version2. You have configured WebSonic to cache the request based on the value of ab-experiment cookie. That means the users with cookie value version1 will be served a different cache content from the users with the cookie value version2. In cases where the content is not found in the cache, the request is proxied to the backend.

In case of a cache miss, the backend receives the request along with the cookie i.e. ab-experiment=version2, now you can serve the content which belongs to this group i.e. version2.

Here is how the configuration should look like. We have configured WebSonic to cache separate content based on the value of ab-experiment cookie.

Subsequent requests from an old user will match with this second rule

We have also configured WebSonic to cache subsequent requests by using a configure cache-control handler.

How to vary content based on cookie value?

We will need to configure WebSonic to cache content separately based on the value of the cookie ab-experiment.

To configure this, we will use a custom caching setup on CDN handler.

Vary content based on cookie value