Whitelist

A major requirement of MDS is to remain extremely malleable when it comes to handling client queries. More specifically, because MDS is deployed and utilized in many unpredictable scenarios, it must be able to be filtered, sorted and projected at the will of the client developers, whilst remaining high performing and scalable.

However, this same flexibility breaks some good design practices such as encapsulation, and as a result leaves us wide open to malicious attacks. An example being something like:

/metadata/delivery/CMS4X/btv/services?filter={"$where":"sleep(10000)"}

A malicious user could easily cause thousands of these queries and swamp the server with unresponsive calls, exhausting connections pools, and causing general disaster. For example, this should do the job:

CODE

while true
do 
  curl -g -X GET 'http://<yourhost>/metadata/delivery/CMS4X/btv/services?filter={"$where":"sleep(5000)"}' &
done

After a few seconds, any normal query responds with 503 Service Temporarily Unavailable. Therefore, the purpose of this configuration is to provide a per-API configurable white list to prevent this kind of situation. The whitelisting feature can be disabled through the appconfig.ini property ClientInterfaceConfig.enforceWhitelist.

The definition of each API white list is described below.

API configuration

enforce

This is a boolean property that switches on the white list for a single API.

maxReturn

This provides the maximum number of records that are allowed to be asked for in the scope of one query. For limit=N, if N is greater than maxReturn then the query will be refused.

allowed

This is a list of query signatures that are allowed for an API. There are several components that qualify a query as a match for a given signature.

All keys of the signature must match those of the query, including any keys of sub-documents of the query, for example.

Query	Signature	Allowed?
{ "serviceRef": "BBC One", "period.start": { "$gte": 1000 } }	{ "serviceRef": string, "period.start": { "$gte": number } }	Yes
{ "serviceRef": "BBC One" }	{ "serviceRef": string, "period.start": { "$gte": number } }	No
{ "period.start": { "$gte": 1000 } }	{ "serviceRef": string, "period.start": { "$gte": number } }	No
{ "serviceRef": "BBC One", "period.start": { "$lte": 1000 } }	{ "serviceRef": "BBC One", "period.start": { "$gte": 1000 } }	No

All keys of the query must occur in a signature (no additional fields queried), for example:

Query	Signature	Allowed?
{ "serviceRef": "BBC One" }	{ "serviceRef": string }	Yes
{ "serviceRef": "BBC One", "period.start": { "$gte": 1000 } }	{ "serviceRef": string }	No

All value types must match those of the signature, for example:

Query	Signature	Allowed?
{ "serviceRef": "BBC One", "period.start": { "$gte": 1000 } }	{ "serviceRef": string, "period.start": { "$gte": number } }	Yes
{ "serviceRef": "BBC One", "period.start": { "$gte": "thousand" } }	{ "serviceRef": string, "period.start": { "$gte": number } }	No
{ "serviceRef": "BBC One", "period.start": { "$gte": 1000 } }	{ "serviceRef": string, "period.start": object }	Yes*

* Here there is a successful match but some leakage is allowed through the white list. Any construct could be inserted at the point of "object", and again could be used for malicious attack. It is best to fully define all signatures down to the level of primitive types, if at all possible.

The types allowed in the value position of a signature must correspond to one of (the standard JSON types):

string
number
boolean
array
object

Default locale

Many of the APIs require the mandatory locale field. If this is not provided by the client then a default will be provided internally by MDS. In either case it is seen that the query includes the locale field, and so this must in turn be included within the query signature.

The following APIs have mandatory locales:

/btv/editorials
/btv/products
/btv/programmes
/btv/series
/btv/services
/vod/editorials
/vod/nodes
/vod/products
/vod/series

Locales are not mandatory for the following APIs:

/btv/version
/vod/images
/vod/promotions
/vod/version
/offers/promotions
/ping
/versions