Tag: Out-File

Error “The ‘<‘ operator is reserved for future use.” (SOLVED)

Analog “<” for PowerShell

On Linux, you can use the following construct:

COMMAND1 < FILE1

In this case, COMMAND1 will be executed with FILE1 as the input source instead of the keyboard, which is the normal standard input source.

The “<” operator corresponds to the use of “|” to be passed to standard input. For example, the following commands are identical:

COMMAND1 < FILE1
cat FILE1 | COMMAND1

Trying to use this construct in PowerShell throws an error.

For example command

mysql -uroot < C:\Users\MiAl\Downloads\all-databases.sql

ends with the following message:

ParserError:
Line |
   1 |  mysql -uroot < C:\Users\MiAl\Downloads\all-databases.sql
     |               ~
     | The '<' operator is reserved for future use.

Similar error in PowerShell 5:

string:1 character:14
+ mysql -uroot < C:\Users\MiAl\Downloads\all-databases.sql
+              ~
The '<' operator is reserved for future use.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : RedirectionNotSupported

Instead of syntax

COMMAND1 < FILE1

you need to use the following structure:

Get-Content FILE1 | COMMAND1

The Get-Content cmdlet will read the contents of FILE1. Symbol “|” (pipe, conveyor) means to pass the content to COMMAND1.

Thus, instead of

mysql -uroot < C:\Users\MiAl\Downloads\all-databases.sql

you need to use the following command:

Get-Content C:\Users\MiAl\Downloads\all-databases.sql | .\mysql -uroot

“./program < input.txt > output.txt” alternative for PowerShell

Consider the construction

./program < input.txt > output.txt

It means that the contents of the input.txt file are passed to the standard input of the “program” command, and the result of program execution is redirected to the output.txt file. But the above command will not work.

An analogue of the considered construction, which will work in PowerShell, is the following command:

Get-Content INPUT.txt | ./program > output.txt

Or you can use the PowerShell style variant:

Get-Content INPUT.txt | ./program | Out-File output.txt

Output encoding issues in PowerShell and third-party utilities running in PowerShell (SOLVED)

What encoding is used in PowerShell by default. How to change the default output encoding to UTF-8 in PowerShell

If you run the following command in PowerShell 5:

"Testing" > test.file

And check the encoding in the newly created test.file, it turns out that it is UTF-16LE.

If you run the following command in PowerShell 7:

"Testing" > test.file

And check the encoding in the newly created test.file, it turns out that it is UTF-8.

The following command, executed in PowerShell 5:

.\mysqldump.exe -u root --all-databases > all-databases_ps5.sql

Creates a UTF-16LE encoded all-databases_ps5.sql file into which all MySQL databases will be exported using mysqldump (for backup purposes). In these databases, ALL non-Latin characters will be irretrievably corrupted!

That is, instead of Cyrillic, it will be something like this:

'╨Р╤А╨▒╨╕╤В╤А╨░╨╢╨╜╤Л╨╣ ╨┐╤А╨╛╤Ж╨╡╤Б╤Б: ╤Г╤З╨╡╨▒╨╜╨╕╨║ / ╨Ъ.╨Ь. ╨Р╤А╤Б╨╗╨░╨╜╨╛╨▓, ╨Ф.╨е. ╨Т╨░╨╗╨╡╨╡╨▓, ╨а.╨Э. ╨У╨╕╨╝╨░╨╖╨

The following command, executed in PowerShell 7:

.\mysqldump.exe -u root --all-databases > all-databases_ps7.sql

Creates a UTF-8 encoded all-databases_ps7.sql file into which all MySQL databases will be exported using mysqldump (for backup purposes). But in these databases, ALL non-Latin characters will be irretrievably corrupted AGAIN!

That is, it would seem that the default encoding has changed to UTF-8, but the problem with completely corrupted database backups has not gone away.

How to save output from third-party programs in UTF-8 encoding in PowerShell

The above behavior, which corrupts the output of commands in PowerShell, is not acceptable. Let's look at how to ensure that output in PowerShell is saved in UTF-8 encoding.

How to save PowerShell output in UTF-8 encoding, use the Out-File cmdlet

Consider the following command:

.\mysqldump.exe -u root --all-databases > all-databases.sql

As already shown above, it corrupts non-Latin characters due to incorrect encoding.

In some sources, as a solution to the problem, it is recommended to replace the “>” character with “Out-File”, and also specify the encoding using the “-Encoding UTF8” option. That is, use the following command:

.\mysqldump.exe -u root --all-databases | Out-File -Encoding UTF8 all-databases_fixed.sql

In fact, both in PowerShell 7 and in PowerShell 5, this command changes little. First, the command is analogous to the output redirection symbol “>”. Secondly, by default, the Out-File cmdlet uses UTF-8 encoding, that is, it is not necessary to specify it specifically.

But the most important thing is that despite the fact that the data is saved to a file with UTF-8 encoding (in the previous command, this is the all-databases_fixed.sql file), non-Latin characters in this file are still corrupted! The thing is that the Out-File cmdlet initially processes the received data in the wrong encoding. Therefore, it no longer matters how exactly Out-File saves the data – the data is corrupted already at the time of receipt in this cmdlet.

The problem was solved by explicitly specifying the encoding for the received data:

[Console]::OutputEncoding = [System.Text.Encoding]::GetEncoding("UTF-8")

The subsequent run of the command saved all the data in the correct encoding:

.\mysqldump.exe -u root --all-databases | Out-File -Encoding UTF8 all-databases_fixed.sql

This method works equally well in PowerShell 7 and PowerShell 5.

By the way, the original command using this method also saves the data in the correct encoding:

[Console]::OutputEncoding = [System.Text.Encoding]::GetEncoding("UTF-8")
.\mysqldump.exe -u root --all-databases > all-databases.sql

Saving output by third-party utilities

In all previous commands, we redirected the output of the mysqldump utility to a file, or to the Out-File cmdlet. But the mysqldump utility has a --result-file option after which you can specify a filename to output. That is, as a result of using this option, you do not need to use output redirection or PowerShell cmdlets.

The following command will save the database in the correct encoding:

.\mysqldump.exe -u root --all-databases --result-file=all-databases.sql

Loading...
X