User Defined Entities

As an API.AI super user, you already understand the power behind the platform. Your implementation is reaching its final stage, and you have started to ask for more advanced capabilities. Most frequently, developers just like you are interested in being able to customize entities on the user/request level.

Let’s take a look at a quick scenario. You are a developer with an application to control all of the devices and settings in your smart home. Out of the box a user can refer to their objects in generic ways like “turn the lights on in my office”. But - what if the user wants a more personalized experience? If this is the case, the end user must be able to add their own names to refer to specific objects. For example, rather than referring to a generic office entity, they could say “turn on the lights in my mancave”.

In order to work with user entities:

1) Create an entity (e.g. @houselocation and @device entities for a smart home app) Populate the entity with default values (e.g. @houselocation has:Kitchen, Living Room, Bedroom and @object has Lights)

2) Create an intent that uses this entity (e.g. “Turn on the @device in the @houselocation”)

3) Utilize the new entities parameter in the query endpoint. The entities object accepts an array of entities, following the format used here.

The submitted entities will replace entities defined in the developer console just for this one request.

Here’s the curl example:

The specified entity needs to exist in the developer console. The UI does not allow for creating an empty entity, so you must have some default value there.

On the Android SDK level: To use the SDK please use the directive below instead of pulling it from the repo:

Usage example:

Finer grain controls for the entities are coming soon.

Speech Interface: The Remote You Can’t Lose

There’s a word for having a remote control for every possible electronic in your life – convenient. Whether you’re settling down to finish some work or tucked in for the night, remote controls add even more convenience to the technology they’re paired with. Until you lose them, their batteries die, or one system requires 3+ remotes just to turn it on. Remotes were the solution for the 20th century, but now we have something better: speech interface.

What could be simpler than saying what you want to happen and then it happening? Let’s look at the - inevitable - smart home use case. We like the different control panels and various apps for each type of appliance we want to control, but what we would love, what would thrill our socks off, is to just say, “turn off the lights and lock the doors,” from our beds.

This is not a science fiction pitch for future generations. You can enjoy this first-class service today with products like the Ubi and Smarter Shade. Imagine walking into your home and saying, “Turn off the garage light, preheat the oven to 375, and turn on the SF Giants game,” and having the relevant electronics comply with your requests. That would be the real smart tv we’ve been waiting for. Speech interface is beyond convenient – it’s efficient and offers a whole new peace of mind. Try out today to make your all in one remote that’s handsfree and convenient.

Voice Enabling TaskList App with

To help you get started in voice enabling your apps for Android, we have put together a basic implementation walkthrough. For this demonstration, we have decided to start with an open source to do list app and turn it into a smart task list that people can easily interact with on the go.

Link to android SDK: HERE

Playstore link to app we will voice enable: HERE

Link to git for TaskList open source code: HERE

Domains Release: Don’t Reinvent the Wheel

If you are looking for quick, out-of-the-box speech to text functionality – then we have news for you! At, we understand that sometimes you don’t want to create everything from scratch and we have the full voice interface solution.

Through the half a decade that we spent creating our nlp engine, we have also created many significant knowledge domains to support interaction within the Assistant. For your convenience, we are releasing these Domain Knowledge Bases into

Domains are pre-defined knowledge packages. By enabling Domains on the appropriately named tab in the dev console, you can make your agent understand thousands of diverse requests – no coding or thinking required! This week we have released Domains for: Smart Home, Maps and Points of Interest, Booking, Media, Times and Dates, Web Browsing, Small Talk, Apps, and Device Control. Of course, more will be coming soon.

So, what does that mean? Now, user’s requests are sent for processing to both your agent and the Domains Knowledge Base. For ease of development, you will see both results in the test console. In the runtime, your agent has a preference. If a request is made via the HTTP API or one of the SDK helpers, will return the agent’s response if available. If not, a Domain response will be returned. If your app requires more commands to be processed within certain Domains, you can create your own intents in addition to what is available. [Note: You need to use the same name for the parameters to be consistent with the parameters returned from the Domains].

This means that you could create an empty agent, and as long as Domains are turned on it will understand all sorts of things (e.g. “I want to hear the Red Hot Chili Peppers”, “Lock the front door at 9 pm”, “Book me a hotel for 2 nights in San Francisco”, “Wake me up at 7:30 am”, “I want a hot dog”, “I want to go to a concert tomorrow”).

For more information, check out our Domains documentation here.

Why not interact with advertisements?

Do you remember the “olden days” when we had to watch TV commercials? We zoned out during irrelevant radio promos, muted Spotify blurbs, and desperately pressed “skip” to YouTube ads. Oh wait – it’s still that way. Digital advertising is a multi-billion dollar industry and yet it is in need of some top-notch innovation. They really haven’t had many options, until now…

With the capabilities of speech to text growing, modern advertisements are on the brink of a revolution. It will leverage both big data and cutting edge technology to provide a personalized ad experience that was previously unimaginable. Paid ads and freeads alike can now be updated dynamically to reflect your preferences and interact with you as if you were, well, a human.

Let’s look at a few use cases through something like Spotify.

Ad: “Hi Mike! I noticed that your mom’s birthday is around the corner.  She might enjoy a dozen hot pink roses.  Would you like me to send her some?”

Mike: “Oh, I forgot.  Please send her two dozen white roses.”

Ad: “Sure thing.  $62 will be charged to your credit card on file.”

Ad: “The summer is creeping up and I have some hot deals specifically for you, Mike.  Would you like to hear more?”

Mike: “Are there any flight to Hawaii deals?”

Ad: “Let me look. Yes, I have a deal for a flight to Maui during the first week of June.  Would you like to book it now or should I send you an email with the details?”
Mike: Send me an email.

Ad: “Hi Mike, Geico can save you 15% in under 15 minutes. If you would like to hear more about how to save on your auto insurance by switching to Geico, just ask”

(Later that day)

Mike: “Can you tell me more about the Geico deal from earlier?”

Ad: “Sure…Let me connect you to a Geico representative to find a plan that’s best for you”