powershell - Block of code prevents script from running, but runs interactively

Question

I have a script that is run as a scheduled task which fails with an unexpected token error on on the line where $As is defined. If I remove the code, the script runs properly. If I paste the whole script (including the problematic section) into a PowerShell window everything runs as expected.

I'm assuming this is a simple gotcha that I've just not encountered, but I cannot figure out what the problem is with it, more experienced eyes would be appreciated.

This is being run on Server 2012R2, with PS 5.0.117 but also happened under version 4.

# Sanitize $UserLogon
$Garbage = "[?' ]",''
$As = '[?àá?????àáa????]','a'
$Cs = '[???]','c'
$Es = '[?èéê?èéê?]','e'
$Is = '[?ìí??ìí??]','i'
$Ns = '[???]','n'
$Os = '[?òó????eòó????]','o'
$Ss = '[??]','s'
$Us = '[?ùú?üùú?ü]','u'
$Thorns = '[?Tt]','th'

$TextReplacers = $Garbage, $As, $Cs, $Es, $Is, $Ns, $Os, $Ss, $Us, $Thorns

foreach ($Replacement in $TextReplacers) {
    $UserLogon = $UserLogon -replace $Replacement
    }

The exact error I receive is:

At C:ScriptsOnboardingCreateUserAccount0.ps1:121 char:17
+     $As = '[??€????????…???????￠?￡?¤?￥?|]','a'
+                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Unexpected token '?????…???????￠?￡?¤?￥?|]','a'
    $Cs = '[????§]','c'
    $Es = '[????‰?????¨???a??]','e'
    $Is = '[??????????-???ˉ]','i'
    $Ns = '[??‘?±]','n'
    $Os = '[??’?“?”???–???°?2?3?′?μ????]','o'
    $Ss = '[???]','s'
    $Us = '[??????????1?o????]','u'
    $Thorns = '[?????]','th'

    $TextReplacers = $Garbage, $As, $Cs, $Es, $Is, $Ns, $Os, $Ss, $Us, $Thorns

    foreach ($Replacement in $TextReplacers) {
        $UserLogon = $UserLogon -replace $Replacement
        }
# Check if AD user already exists.
$UserExists = Get-ADUser -Filter {SamAccountName -eq $UserLogon}
if ($UserExists -ne $Null){
    $email = new-object Net.Mail.SMTPClient($mailServer)
    $err += "$UserLogon' in expression or statement.

If I comment out the $As, it happens with $Ns, and $Os. If I comment out $As, $Ns and $Os, it runs fine.

Answer 1

PowerShell can detect following encodings from script file BOM: UTF-8, UTF-16 (LE and BE) and UTF-32 (LE and BE). If BOM is not present, then PowerShell use Encoding.Default for script file. So that, your UTF-8 script file should include BOM for UTF-8 to be recognized.

In your case, error happens due to PowerShell interpret all following characters: '‘’?? — as single quote character. So, when your script file was read with incorrect encoding, some parts of what was string literals obtain special meaning and cause syntax violation.

$As = '[??€????????…??? ???￠?￡?¤?￥?|]','a'
             ^
$Ns = '[??‘?±]','n'
          ^
$Os = '[??’?“?”???–???°?2?3?′?μ????]','o'
          ^