Subscription filters

  • Updated: Sep 2, 2016
  • Starting Guide
  • Pub/Sub
  • Filters

Sometimes when you subscribe a channel you wish the Realtime server could filter the messages sent to you based on the message content.

This would save you money on Realtime messages while saving your user's battery and bandwidth.

The new subscription filter allows you to achieve that!

How?

Imagine you are subscribing a weather channel where several sensors publish data about the location and temperature with the following JSON format (temperatures in Fahrenheit):

{
	"city": "London",
	"temperature": 50.8,
	"timestamp": 1472464189645
}
        

If you subscribed the channel without any filter you will receive every message, even if they are about a location that you're not interested in, forcing you to discard these messages on the client side.

Now with the subscription filter you can pass a filter condition in SQL WHERE format.

The Realtime server will evaluate the filter for each message published in the channel and will only deliver the messages that validate your filter condition.

An example? Sure!

Let's say you are only interested in London temperature readings. To filter the messages you would send the following filter when you subscribe the channel:

message.city = 'London'
        

Pretty simple, right?

But it gets better! Since the filter is a SQL formated condition you can add multiple conditions to your filter, like the temperature reading value itself.

Let's say you are not only interested in London temperatures but also only interested in temperature readings above 77°F - maybe you would like to send your London users an alert to buy your ice-creams!

The new filter condition would be as follows:

message.city = 'London' and message.temperature > 77
        

With this filtering in place your clients would receive less messages - only those they are interested in - and this would lead to battery and bandwith savings while reducing your Realtime costs (we don't charge for the messages that are not delivered to subscribers).

Usage example

The following JavaScript snippet shows how to use the new subscribeWithFilter method to subscribe a channel using the subscription filter referred in the previous section:


// the subscription filter to be applied
var filter = "message.city = 'London' and message.temperature > 77";

// subscribe the channel using a filter
ortcClient.subscribeWithFilter('myChannel', true, filter, 
	function(ortc, channel, filtered, message) {
		// message received from server
    	console.log("Message successfully filtered by the server:", filtered);
    	console.log("Message received:", message);
  	});
	

As simple as that!

Notice the subscribeWithFiltercallback has an additional parameter filtered that is not present in the regular subscribe method.

This boolean parameter allows you to know if the message being delivered was successfully filtered by the server. In the event the message was not filtered by the server you should be able to evaluate the message on the client or simply discard it. See the limitations section below for more details.

Now copy the following complete code to an HTML file and open it in your browser to test the filter subscription (don't forget to replace INSERT_YOUR_APP_KEY_HERE with your Realtime Messaging application key):

<html>
<head>
	Realtime JavaScript Subscription Filters
	

	

</head>
<body>
</body>
</html>
		

Let's send some messages to see it working.

Open your Realtime Console and send the following messages to channel myChannel:

{
	"city": "London",
	"temperature": 80,
	"timestamp": 1472464189645
}
        
{
	"city": "Paris",
	"temperature": 70.8,
	"timestamp": 1472464189645
}
        

Inspecting your browser log will show that you have received the first message but not the second, since only the first one is valid according to the subscription filter.

Nice!

Albeit the example above is coded in JavaScript, the subscription filters are already available in the lastest versions of some Realtime SDKs (see the Limitations section below). Check the SDKs reference guides for more details about the new subscribeWithFilter method.

The filter format

The Realtime subscription filter implements a sub-set of the SQL WHERE format with the following specification:

Operators

  • =
  • >=
  • >
  • <=
  • <>
  • <
  • like

Conditionals

  • and
  • or

Math

  • +
  • -
  • *
  • /

Note: The conditionals and like operator are case-sensitive, you must use them in lower-case. You must also use ' as the text delimeter.

Data scope

As you have seen in the examples above, the message published will be available to your filters in the message namespace.

If your message is JSON formatted you can access your JSON object properties using the . notation, incluing nested objects.

If our temperature example messages also included the latitude and longitude as a geo object property as follows:

{
	"city": "London",
	"temperature": 50.8,
	"timestamp": 1472464189645,
	"geo": {
		"latitude": 51.528308,
		"longitude": -0.3817802
	}
}
        

We could use the latitude property in our filters as shown in the filter below:

message.geo.latitude > 50 and message.temperature > 77
        

Notice the message.geo.latitude > 50 condition as an example of using nested objects.

Non-JSON messages

If your messages are not in JSON format you'll be more limited in your conditional options but you'll still be able to use the subscription filters.

In this case the message namespace will contain your message as a string and you could apply the LIKE operator as follows (to get only the messages with at least two parts separated by a colon mark:

message LIKE '%:%'
        

Notice the % wildcard used in the like operator.

Invalid filters

If you use an invalid filter for your subscription you'll get a "Subscription filter is not valid" exception as soon as the first message is published in the channel.

What if messages are encrypted?

If the messages you are sending through Realtime are encrypted, our servers won't be able to apply the filtering because they can't inspect their content.

But there's a quick and easy solution: adding metadata to the original message.

If you send your messages with two properties, data and metadata, you can still encrypt the data property (the message) and enter the properties through which you want to filter in the metadata property with no encryption. See the example below:

{
	"data": "LwXZGrtHDTdATQrcFjIp6w==",
	"metadata": {
		"senderid": "699c5236-280e"
	}
}
        

With the message format above you could use the following subscription filter while still keeping your data encrypted:

message.metadata.senderid <> '699c5236-280e'
        

For security and privacy reasons it is critical to only include in the metadata information that is not confidential.

Limitations

Filter length

Your subscription filters are limited to 1024 characters. Using a filter above this limit will result in a "Subscription filter is too long" exception.

Message length

The subscription filter will only be applied to messages smaller than 8Kb. Larger messages will be delivered as in a regular subscription with the callback parameter filtered set as false, meaning the message was not filtered by the server. In this scenario you'll have to filter the message on the client side.

Filter timeout

If the rare event of the messages queued for delivery being too large causing increased latency (above 5 seconds), the Realtime server will temporarily suspend the filter evaluation and deliver the messages as in a regular subscription with the callback parameter filtered set as false, meaning the message was not filtered by the server. In this scenario you'll have to filter the message on the client side.

Supported Realtime SDKs

Currently only the following Realtime SDKs implement the subscribeWithFilter method (please let us know if the SDK you need is not yet in this list):

  • JavaScript 2.1.30+
  • Node.js 2.1.24+
  • Android (Java) 2.1.66+
  • iOS (ObjC) 2.1.30+
  • iOS (Swift) 2.1.2+
  • Java 2.1.40+
  • React Native Android 1.1.2+
  • React Native iOS 1.0.16+

Back to Security - Next: Delivery modes

If you find this interesting please share: