New M function: Text.ReplaceAll

Replacement dictionary for Text.ReplaceAll

Update 4 June 2019: unfortunately, the function was not meant to be published. The function is part of a certified connector and is now hidden.

The May 2019 release of Power BI Desktop includes a new M function: Text.ReplaceAll (not documented on Microsoft Docs as of 18 May 2019). This function simplifies multiple word replacements that previously had to be done by using custom functions.

Background

Replacing multiple text strings at once by using Power Query is a common scenario. Several high-profile bloggers have written their own functions to perform multiple replacements — Chris Webb, Ivan Bondarenko and Imke Feldmann, to name a few (Imke’s formula is slightly different — more on that later).

In this blog post, I’ll be using the same data as they did:

Text query
Text
the cat sat on the mat
the cat sat next to the dog
the dog chased the cat
the dog sat on the mat
let
    Source = #table(
        type table [Text = text],
        {
            {"the cat sat on the mat"},
            {"the cat sat next to the dog"},
            {"the dog chased the cat"},
            {"the dog sat on the mat"}
        }
    )
in
    Source
Replacements query
Word to Replace Replace With
cat bear
mat chair
dog dragon
the THE
let
    Source = #table(
        type table [
            Word to Replace = text,
            Replace With = text
        ],
        {
            {"cat", "bear"},
            {"mat", "chair"},
            {"dog", "dragon"},
            {"the", "THE"}
        }
    )
in
    Source

Function syntax

Text.ReplaceAll takes two arguments:

  1. Text
  2. Replacements

The second argument is a list of pairs of “from” and “to” values.

For example, the following expression returns “yyz”:

Text.ReplaceAll("xyz", {{"x", "y"}})

Using Text.ReplaceAll

To use the function on Chris’s data, we need to convert the dictionary to a list of lists first. For that, we can use List.Zip. Assuming our dictionary table is in a step called Source, the following expression will give us exactly what we need:

= List.Zip({Source[Word to Replace], Source[Replace With]})

So the whole Replacements query then becomes:

let
        Source = #table(
        type table [
            Word to Replace = text,
            Replace With = text
        ],
        {
            {"cat", "bear"},
            {"mat", "chair"},
            {"dog", "dragon"},
            {"the", "THE"}
        }
    ),
    ToList = List.Zip(
        {
            Source[Word to Replace],
            Source[Replace With]
        }
    )
in
    ToList

Applying the function is then straightforward. If we add a custom column, this is the formula we can use:

= Text.ReplaceAll([Text], Replacements)

Here are the results:

Text.ReplaceAll replaces all strings with replacement strings

Considerations

It is important to note that the function currently only replaces substrings, not whole words. I.e., if you introduce an extra row to the Replacements table like so:

Word to Replace Replace With
cat bear
mat chair
dog dragon
the THE
air water

The results will probably be not what you want:

Text.ReplaceAll does not have an option to replace whole words only

In case you need to replace whole words only, I would use Imke’s approach, which addresses the issue.

Note that the function performs replacements sequentially: if you place the air-water pair at the top of the list, you will get different results.

You can download my .pbix file here: TextReplaceAll.pbix

  • Thanks for the update, as for replacing whole words I actually really like using the Record.FieldOrDefault method that Curt Hagenlocher proposed back in 2015.
    https://social.technet.microsoft.com/Forums/windows/en-US/dc601091-f98f-48cf-b6ce-af1342833419/replace-multiple-values-in-a-single-step?forum=powerquery
    You can choose a default value if no match is found (e.g. could be the current value or null) and can also be used as a way of joining if used with Table.AddColumn

    • Thanks for sharing, Cameron! It seems like there’s a significant difference between Imke’s and Curt’s approaches: Imke replaces whole words (by words I mean words, not whole sentences), while Curt replaces whole cell values (in this example that would be sentences). So if you apply Curt’s formula to Imke’s data, the output will be the same as input. Curt’s formula is valuable when you need to operate on whole strings/cells.

  • Hossein SATOUR

    Thank you for this article.
    I tried to use this function today, June 2nd 2019, with the last version of Power BI, but the function wasn’t recognized by the it. Perhaps it was deleted !

    • Hossein, you are correct — unfortunately, it has become unavailable because it was not meant to be published in the first place.

      • Hossein SATOUR

        Thank you Daniil. But the strange thing is while I installed the May version of Power BI, the function was in it, few days later, without reinstalling PBI, the function disappeared, how it is possible ?

        • Either Microsoft is able to remotely change things in Power BI or you have the Windows Store version — is it the latter?

          • Hossein SATOUR

            Many thanks Daniil. As I said, it’s very strange, I saw the function in the last version of Power BI (May release, not from Windows store but by downloading it from the PBI website) and it disappeared few days later in the same version (without reinstalling the software)!