You are currently viewing Dialogflow 11 – Playing audio files and SSML

Dialogflow 11 – Playing audio files and SSML

Hi,

One of the options working with Dialogflow and Google Assistant is to use SSML, the HTML of the speach generation languages, to create richer audio content.

Google Assistant supports a subset of the SSML language that you can check here. I’m not going to review all the details and all the commands that would allow you to contro pauses, tone, etc., as you already have the official doc, and as you can see in this example it is pretty straight forward.:

One of the key options is to play audio files … again is it easy and with several supported audio formats. In case you use audio files, please bare in mind the Assistant’s Terms and Conditions … for example you should not imitate any of the Assistant’s voices.

The SSML primitives can be set in the default audio response, but be carefull, because if you call dialogflow from an interface different from Google Assistant it might no be able to render such response. That’s why it is probably better to generate an specific Google Assistant’s  Simple Response with the SSML content.

If we want to do it from a fulfillment function the code would be this one:

Before that you have to define “Texts” as one of the primitives defined in the Dialogflow fulfillment library:

const {Card, Suggestion, Text} = require(‘dialogflow-fulfillment’);

As you can see you can define both the text that is going to be read and the one displayed on the screen.

Finally, audio files should be placed as a Cloud Storate object, as we do with images used as resources for our agents.

Regards,

Jamarmu

============================================================

I work for Google Cloud, but this post are personal ideas and opinions

This Post Has 4 Comments

  1. Ayham 21 Sep 2020 Reply

    hey i am trying to add one second to every element i have and i worte it in this way but it is not working is there a way to solve this problem. and thanks

    let text = new Text(“”);
    let word = [‘kod’ ,’spion’];
    word.forEach(element => {
    text.setSsml(” +
    ” + element +
    ”);
    agent.add(element);
    });
    agent.add(text);

    1. jamarmu 30 Sep 2020 Reply

      Hi Ayham,

      To add the pauses you need to add the following: [break time=”3s”/]

      The full code to program this would be:

      function pruebaSSML(agent) {
      let answer = new Text(“”); `
      answer.setSsml(“[speak] hello [break time=’3s’/] bye[/speak]”);
      answer.setText(“hello bye”);

      agent.add(answer);
      }

      You have to changes de square brackets to angle ones 😀 … sorry the html was having fun for me.

      The simulator won’t display this correctly as it will write the SSML output as the default response, but if you try this with the Assistant integration it works perfectly.

      The reference for SSML is here:
      https://cloud.google.com/text-to-speech/docs/ssml

      I hope it helps

      Jamarmu

  2. ian 27 Oct 2020 Reply

    Hey, nice tutorial, just a quick question:
    Is it possible to send audio payloads to the “Dialogflow Messenger” integration for websites?
    Couldn’t find anything related to this, i think its only possible on other integrations like Facebook.

    Thanks in advance,

Leave a Reply