Tag: Python

How to convert a string to lowercase in Bash

This note will show you how to convert a string to lowercase (small letters) on the Linux command line.

To convert a string to lower case regardless of its current case, use one of the following commands.

tr

echo "Hi all" | tr '[:upper:]' '[:lower:]'
hi all

Attention! If you want to change the case of any letters other than Latin (national alphabets, letters with diacritics), then do not use tr, but use any other solution suggested below. This is because the classic Unix tr operates on single-byte characters and is not compatible with Unicode.

AWK

echo "Hi all" | awk '{print tolower($0)}'
hi all

Bash

a="Hi all"
echo "${a,,}"
hi all

Starting with Bash 5.1, there is a conversion option “L”, which is intended to convert a string to lowercase:

${var@L}

Example:

v="heLLo"
echo "${v@L}"
hello

sed

echo "Hi all" | sed -e 's/\(.*\)/\L\1/'
hi all

Or this solution:

echo "Hi all" | sed -e 's/\(.*\)/\L\1/' <<< "$a"
hi all

Another solution:

echo "Hi all" | sed 's/./\L&/g'

Perl

echo "Hi all" | perl -ne 'print lc'
hi all

Python

a="Hi all"
b=`echo "print ('$a'.lower())" | python`; echo $b

Ruby

a="Hi all"
b=`echo "print '$a'.downcase" | ruby`; echo $b

PHP

b=`php -r "print strtolower('$a');"`; echo $b

NodeJS

b=`node -p "\"$a\".toLowerCase()"`; echo $b

In zsh

a="Hi all"
echo $a:l

How to convert a string to uppercase in Bash

This note will show you how to convert a string to upper case (capital letters, uppercase) on the Linux command line.

To convert a string to capital letters regardless of its current case, use one of the following commands.

tr

echo "Hi all" | tr '[:lower:]' '[:upper:]'
HI ALL

Attention! If you want to change the case of any letters other than Latin (national alphabets, letters with diacritics), then do not use tr, but use any other solution suggested below. This is because the classic Unix tr operates on single-byte characters and is not compatible with Unicode.

AWK

echo "Hi all" | awk '{print toupper($0)}'
HI ALL

Bash

a="Hi all"
echo "${a^^}"
HI ALL

Starting with Bash 5.1, there is a conversion option U, which is intended to convert a string to uppercase:

${var@U}

Example:

v="heLLo"
echo "${v@U}"
HELLO

sed

echo "Hi all" | sed -e 's/\(.*\)/\U\1/'
HI ALL

Or this solution:

echo "Hi all" | sed -e 's/\(.*\)/\U\1/' <<< "$a"
HI ALL

Another solution:

echo "Hi all" | sed 's/./\U&/g'

Perl

echo "Hi all" | perl -ne 'print uc'
HI ALL

Python

a="Hi all"
b=`echo "print ('$a'.upper())" | python`; echo $b

Ruby

a="Hi all"
b=`echo "print '$a'.upcase" | ruby`; echo $b

PHP

b=`php -r "print strtoupper('$a');"`; echo $b

NodeJS

b=`node -p "\"$a\".toUpperCase()"`; echo $b

In zsh

a="Hi all"
echo $a:u

How to get web page content and cookies of .onion sites in Python

A program that receives data from the Tor network must work with cookies, for example, in the case of a parser, it can be cURL, PHP script, Python script, and so on.

In the article “Web site parsing in command line” there is an example of working with cookies in cURL, but how to get the content of a web page (HTML code) and cookies of a Tor network site whose names end in .onion?

For the parser to work with the Tor network, you need to specify the data of the local Tor service (port number and “localhost” as IP) as a proxy for accessing the network.

For normal operation with .onion sites, you need to use the Tor DNS servers.

In the Python script, to access .onion sites, you need to use the socks5h protocol to enable the use of remote DNS to resolve hostnames to IP if local DNS resolution fails.

The following code shows the .onion page of the site (URL http://hacking5xcj4mtc63mfjqbshn3c5oa2ns7xgpiyrg2fenl2jd4lgooad.onion) and cookies:

import requests
import json

proxies = {
	'http': 'socks5h://127.0.0.1:9050',
	'https': 'socks5h://127.0.0.1:9050'
}

session = requests.Session()

data = session.get("http://hacking5xcj4mtc63mfjqbshn3c5oa2ns7xgpiyrg2fenl2jd4lgooad.onion",proxies=proxies).text

print(data)

print(session.cookies)

A simple PHP script is used as a site that sends HTML code and cookies:

An example of how the code above works – you can see HTML and cookies:

Line

print(session.cookies)

Outputs:

< RequestsCookieJar[< Cookie HackWare-cookie=For%20testing%20purpose%20only for hacking5xcj4mtc63mfjqbshn3c5oa2ns7xgpiyrg2fenl2jd4lgooad.onion/ >] >

That is, the format is:

< RequestsCookieJar[< Cookie NAME=VALUE for SITE.onion/ >] >

If print (session.cookies) is changed to

print(session.cookies.get_dict())

then the format will be like this:

{'HackWare-cookie': 'For%20testing%20purpose%20only'}

Basically, sites can encrypt cookies. More precisely, in any case, cookies will be sent in the “NAME=VALUE” format. But the VALUE can be encrypted so that only the site will know what to do with it. But in general, the user does not need to think about it – what cookies were get, those are sent by the browser.

ONVIF client with command line interface

ONVIF is an open industry forum that provides and promotes standardized interfaces for effective interoperability of IP-based physical security products.

ONVIF protocol can be found on security cameras. Using ONVIF protocol, you can receive information from IP cameras and control them (set settings, rotate, etc.).

This article will talk about python-onvif - an ONVIF client implemented in Python. This program can be considered the successor to foscam-python-lib - Foscam Python Library for H.264 IP Cameras (FI9821W/P/HD816W/P).

This instruction will show working examples of using ONVIF on the command line.

To install python-onvif run:

sudo pip3 install --upgrade onvif_zeep

If you get an error that the pip3 command was not found, then you need to install the python3-pip package (for Debian based distributions) or python-pip (for Arch Linux based distributions).

You can find various examples of using python-onvif (in Python code, in scripts, in interactive mode, etc.) on the program home page: https://github.com/quatanium/python-onvif. The documentation is not informative, and due to the fact that you need to specify the wsdl directory, the path to which by default differs from that used in the examples, and also differs from distribution to distribution, all examples turn out to be inoperative.

Therefore, I propose to consider the use of python-onvif using an example of working code.

Create an extractor.py file with the following content:

import sys
from onvif import ONVIFCamera

# check if a user has been specified to connect. If not, use an empty string as the username
if len(sys.argv) < 4:
	user = ''
else:
	user = sys.argv[3]

# check if a password was specified for the connection. If not, use an empty string as a password
if len(sys.argv) < 5:
	password = ''
else:
	password = sys.argv[4] 		

# we create an object indicating the host, port, user, password and path to wsdl
mycam = ONVIFCamera(sys.argv[1], sys.argv[2], user, password, '/usr/lib/python3.9/site-packages/wsdl/')

# request and display information about the device
resp = mycam.devicemgmt.GetDeviceInformation()
print (str(resp))

# request and display information about network interfaces
resp = mycam.devicemgmt.GetNetworkInterfaces()
print (str(resp))

# request the address of the media stream
media_service = mycam.create_media_service()
profiles = media_service.GetProfiles()
token = profiles[0].token
mycam = media_service.create_type('GetStreamUri')
mycam.ProfileToken = token
mycam.StreamSetup = {'Stream': 'RTP-Unicast', 'Transport': {'Protocol': 'RTSP'}}
print(media_service.GetStreamUri(mycam))

Notice the line

/usr/local/lib/python3.9/site-packages/wsdl/

You need to replace it with your own value. This line is fine for Kali Linux. And on BlackArch/Arch Linux you need to use the following line:

/usr/lib/python3.9/site-packages/wsdl/

When the Python version changes, the line may change as well.

The path to it can be found by the following algorithm:

sudo updatedb # Update DB with file information
locate accesscontrol.wsdl # We are looking for a file that is in the desired directory

For example found:

/usr/local/lib/python3.9/site-packages/wsdl/accesscontrol.wsdl

We take the entire line, except for the file name, that is, /usr/local/lib/python3.9/site-packages/wsdl/.

Run like this:

python3 extractor.py HOST PORT [USER_NAME] [PASSWORD]

The script makes three separate requests and displays three groups of data: device information, network interfaces, and media stream.

USER_NAME and PASSWORD are optional - if they are absent, an attempt will be made to connect with empty credentials.

Launch example:

python3 extractor.py 118.39.210.69 80

Output example:

{
    'Manufacturer': 'BOSCH',
    'Model': 'AUTODOME IP starlight 7000 HD',
    'FirmwareVersion': '25500593',
    'SerialNumber': '044123455',
    'HardwareId': 'F0004D43'
}
[{
    'Enabled': True,
    'Info': {
        'Name': 'Network Interface 1',
        'HwAddress': '00-07-5f-8b-5d-2b',
        'MTU': 1514
    },
    'Link': {
        'AdminSettings': {
            'AutoNegotiation': True,
            'Speed': 100,
            'Duplex': 'Full'
        },
        'OperSettings': {
            'AutoNegotiation': True,
            'Speed': 100,
            'Duplex': 'Full'
        },
        'InterfaceType': 6
    },
    'IPv4': {
        'Enabled': True,
        'Config': {
            'Manual': [],
            'LinkLocal': None,
            'FromDHCP': {
                'Address': '118.39.210.69',
                'PrefixLength': 24
            },
            'DHCP': True,
            '_value_1': None,
            '_attr_1': None
        }
    },
    'IPv6': None,
    'Extension': None,
    'token': '1',
    '_attr_1': {
}
}]
{
    'Uri': 'rtsp://118.39.210.69/rtsp_tunnel?h26x=4&line=1&inst=1',
    'InvalidAfterConnect': False,
    'InvalidAfterReboot': True,
    'Timeout': datetime.timedelta(0),
    '_value_1': None,
    '_attr_1': None
}

Manufacturer, MAC address, video URI:

    'Manufacturer': 'BOSCH',
        'HwAddress': '00-07-5f-8b-5d-2b',
    'Uri': 'rtsp://118.39.210.69/rtsp_tunnel?h26x=4&line=1&inst=1',

Now that we know how to create an ONVIFCamera object and perform elementary actions with it, we can return to https://github.com/quatanium/python-onvif for more examples.

Also see:

How to run small Python code in Bash

Bash is not only and even not so much shell built-in functions as command line programs (utilities). By running these commands and transferring the received data to the pipeline (through the pipe), you can automate a variety of things, programming of which in other programming languages can take a lot of effort.

This post will show you how to run Python code in Bash, and how to run a Python program in a Bash script.

How to execute Python code in Bash

On the Bash command line, use the following construct to execute code:

python -c "CODE"

For instance:

python -c "print (\"It works\")"

Another option that may come in handy in more exotic circumstances:

bash -c 'python -c "print (\"CODE\")"'

For instance:

bash -c 'python -c "print (\"It works\")"'

How to run a Python program in a Bash script

To execute a Python program in a Bash script, use a command like:

python SCRIPT.py ARGUMENT1 ARGUMENT2 ARGUMENT3

An example of running the extractor.py script passing two arguments: the value of the $line variable and 8080:

python extractor.py $line 8080

How to run a Python program in a Bash script and assign its output to a variable

If you need to run a Python script, and then assign the program output to a Bash variable, then use a construction like this:

ПЕРЕМЕННАЯ=`python SCRIPT.py ARGUMENT1 ARGUMENT2 ARGUMENT3`

Example:

response=`python extractor.py $line 8080 2>/dev/null | sed -E "s/\/\/.+:/\/\/$line:/"`

How to embed Python code into a Bash script

If Python code cannot be used on one line and you do not want to use an external script to call, then the following construction will work for you:

#!/bin/bash

script=`cat <<_EOF_
CODE
_EOF_`

echo "$script" | python

Where the CODE is, paste the Python code.

This example is working:

#!/bin/bash

value="I will be in the script!"

script=`cat <<_EOF_
print ("It works")
print ("Line 2")
print ("Line 3: $value")
_EOF_`

echo "$script" | python

Another variant of this construction:

python << END
... CODE ...
END

And another option, in which the output is assigned as the value of the ABC variable:

ABC=$(python << END ... END)
Loading...
X