Month: February 2021

Password and unix_socket authentication in MySQL and MariaDB. Error “#1698 - Access denied for user ‘root’@’localhost’” (SOLVED)

How to fix #1698 - Access denied for user ‘root’@’localhost’

In the latest versions of MariaDB (possibly MySQL), unix_socket authentication is used by default. If you are not familiar with it, then you might have encountered the error “#1698 - Access denied for user ‘root’@’localhost’”.

Let's take a look at what unix_socket authentication is in MySQL and MariaDB, and how to fix bug #1698.

MySQL and MariaDB unix_socket authentication

The essence of unix_socket authentication is that if the user has already logged into the system, then he does not need to enter a password when connecting to the DBMS, since his authenticity has already been verified when logging into the OS.

In practice, most people work as a regular user and connect to MySQL as root. As a result, the above error occurs.

You can choose one of the options:

1. Always use sudo when connecting as root.

2. Make changes to the MySQL settings so that ordinary users can connect to the DBMS.

3. Create a MySQL user with the same name as your system username

How to check which authentication method is being used

To view the used authentication method, you can use the following SQL query:

select * from mysql.global_priv where User='root';

Or this, for greater clarity of the output:

SELECT CONCAT(user, '@', host, ' => ', JSON_DETAILED(priv)) FROM mysql.global_priv where user='root';

You can see that mysql_native_password and unix_socket are set as plugin:

{
    "access": 18446744073709551615,
    "plugin": "mysql_native_password",
    "authentication_string": "invalid",
    "auth_or": 
    [
        
        {
        },
        
        {
            "plugin": "unix_socket"
        }
    ]
} 

With this configuration, only unix_socket authentication worked for me.

Enabling and disabling unix_socket authentication

You can switch to password authentication with the following SQL query:

ALTER USER 'root'@'localhost' IDENTIFIED BY 'PASSWORD';

Please note that you need to enter the PASSWORD.

To switch to unix_socket authentication, execute the following SQL query:

ALTER USER 'root'@'localhost' IDENTIFIED VIA unix_socket;

Let's check:

SELECT plugin from mysql.user where User='root';

If mysql_native_password is output, it means that password login is being used.

In fact, unix_socket authentication can be combined with password authentication, but I will not dwell on that.

Replacement for “update user set plugin='' where User='root';”

Previously, a similar effect - changing authentication from unix_socket to password authentication - was achieved using a sequence of commands:

Connecting to MySQL Server:

sudo mysql

At the MySQL prompt, you had to run the commands:

use mysql;
update user set plugin='' where User='root';
flush privileges;
exit

Then the service had to be restarted:

sudo systemctl restart mysql.service

And it was possible to connect without sudo.

mysql -u root -p

In the case shown above, the authentication method was also changed from unix_socket to password, but the new password was not set. If you want the same effect (although it becomes insecure after disabling authentication with unix_socket), then you can run the following requests (i.e. set an empty password):

use mysql;
ALTER USER 'root'@'localhost' IDENTIFIED BY '';
exit

Choosing an authentication method when creating a user

You can create a user with password authentication with an SQL query of the following form:

CREATE USER USERNAME@HOST IDENTIFIED BY 'PASSWORD';

To create a user with unix_socket authentication, execute the following SQL query:

CREATE USER USER@HOST IDENTIFIED VIA unix_socket;

How to Run a Program Automatically on Startup in Linux

If you want a program or script to run when the system starts up, then this can be done using systemctl. This method is universal: autorun also works on headless servers, not just when you enter a graphical desktop environment. This method is universal and will work on all systems where systemctl is present (e.g. Debian, Linux Mint, Ubuntu, Kali Linux, Arch Linux, etc.).

To add a script to autoload in Linux, you need to create a specific file in the /etc/systemd/system/ directory. Choose your own file name. For example, I want to run a script on startup that contains several rules to hinder DOS attacks. This file is located at /root/firewall.sh

We start by assigning the correct permissions to the file:

chmod 755 /root/firewall.sh

Make sure the file has shebang #!/bin/bash (or whatever matches the contents of the file).

I create a file anti-dos.service (you can choose your own name):

vim /etc/systemd/system/anti-dos.service

Content of my file

[Unit]
Description=Initial anti-DOS protection.

[Service]
ExecStart=/root/firewall.sh

[Install]
WantedBy=multi-user.target

Here

  • Description=Initial anti-DOS protection. - this is a description, replace the description with your own.
  • ExecStart=/root/firewall.sh - full path to the file that I want to add to startup is specified.

Don't write something like “ExecStart=/bin/sh /path/to/script.sh” as that won't work. If you really want to specify an interpreter and a script, then use a construction like this:

/usr/bin/bash -c 'INTERPRETER /PATH/TO/SCRIPT'

For instance:

/usr/bin/bash -c 'php /root/bin/translator.php'

Example file using this construction

[Unit]
Description=Launch translator.
 
[Service]
ExecStart=/usr/bin/bash -c 'php /root/bin/translator.php'
 
[Install]
WantedBy=multi-user.target
  • WantedBy=multi-user.target - means that autorun is made for all users.

To start the services in the current session (change the name anti-dos.service to the name of your file):

systemctl start anti-dos.service

To check the status of a service:

systemctl status anti-dos.service

To enable autostart, you need to do this (change the name anti-dos.service to the name of your file):

systemctl enable anti-dos.service

Creating a link says that adding to autostart worked:

Let's restart the server to check))

The screenshot above shows that everything worked properly. The fact that the process is shown as deceased, in my case, is normal. Since I added not a service to autostart, but a one-time script.

To make sure that the firewall settings are in effect, I type:

iptables --list

Yes, my firewall rules are there.

You can also find your autorun file in the list of everything added to startup:

systemctl list-unit-files

After editing files with the extension .service, for the changes to take effect, you need to run the command:

systemctl daemon-reload

Error “Unable to load dynamic library ‘xmlrpc.so’” (SOLVED)

When running a web server or running PHP scripts on the command line, you may encounter an error:

PHP Warning: PHP Startup: Unable to load dynamic library 'xmlrpc.so' (tried: /usr/lib/php/modules/xmlrpc.so (/usr/lib/php/modules/xmlrpc.so: cannot open shared object file: No such file or directory), /usr/lib/php/modules/xmlrpc.so.so (/usr/lib/php/modules/xmlrpc.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0

In fact, this is not an error at all, but just a warning. That is, the server and PHP scripts should still work fine.

This warning says that when PHP starts up, the dynamic library 'xmlrpc.so' (PHP module) cannot be loaded. An attempt was made to download this file from all typical locations, but could not be found anywhere.

XMLRPC is an extension that was included in PHP that brought XML RPC server and client functions to PHP.

This extension was relatively unused and was labeled “experimental” all along. This extension relied on some libraries that had not been supported for several years.

The xmlrpc extension is no longer bundled with PHP since PHP 8.0. You can still install the extension from PECL if your code or any dependencies require it.

If you do not need this extension, then in the php.ini file find the line

extension=xmlrpc.so

and comment it out to get:

;extension=xmlrpc.so

If you really need this extension, then install it from PECL, or use one of the following alternatives:

What is suip.biz website? Analogs and mirrors suip.biz

Site suIP.biz

SuIP.biz (short for “super IP”) started out as a service with several pages for various actions with IP addresses:

  • compilation of IP ranges of countries, cities, Internet providers
  • getting information about IP and IPv6 address (location, ISP, whois)
  • showing the user his IP

Subsequently, the site acquired calculators and converters for IP and IPv6 addresses:

Even when suIP.biz was created, services were added to scan IP addresses to find open ports.

Services for collecting information were gradually added:

In addition to the listed services, the site has the following sections:

  • Vulnerability scanners, open ports and running web server services
  • Subdomains and hidden files
  • Obtaining information on MAC addresses
  • Analysis of the web server
  • Working with hashes
  • Analysis of emails
  • Analysis of executable files
  • Extract information from cache and web archives
  • Bypassing the restriction on displaying source HTML code, bypassing social lockers
  • Advanced use of search engines
  • Working with encodings
  • CloudFlare Counter Tools
  • Images and metadata
  • Information about phone numbers

There are several services in each of these sections!

In general, currently the site suIP.biz has grown quite a lot. About a thousand people from different countries use it daily.

Analogs and mirrors suIP.biz

Due to the peculiarities of the suIP services, which for the most part are Linux utilities that run on the server and work for some time, due to the increased load, the suIP.biz site may be offline for some time. The site itself monitors its “health” and to prevent critical failures, it can interrupt running tasks itself. But this does not always help, because the site runs on a regular VPS hosting and is a hobby of the sole author. Therefore, work breaks reach hours in the worst situations.

If for any reason you cannot use the services of suIP.biz, then refer to the services of the site w-e-b.site - this is a similar service from the same author, located on a different server. Since w-e-b.site is aimed at English-speaking users (and the server is hosted in the USA), this site has only an English-language interface.

Reviews suIP.biz

You can leave a review and find out the latest news about the work of suIP on this page.

How to add PHP path to %PATH% environment variable on Windows

What is PATH and what is it for

It is quite possible that you have never encountered echo %PATH% and environment variable expressions before, so I will briefly explain what it is.

The PATH variable contains a list of folders in which Windows looks for executable files.

In the GUI, when shortcuts are used to launch programs, the PATH value is not very large. But if you run the program on the command line, then PATH can come in handy. Again, if you specify the full path to the file, for example, C:\Users\Alex\Documents\php.exe, then PATH is not used. But if, for example, you run the program only by the file name or just by the name (without the file extension), then whether the program starts will depend on the contents of the PATH variable.

For example, I try to run a file on the command line (without specifying the full path)

php.exe

In this case, the operating system will look at all PATH entries (several directories may be specified there). Windows will then try to find the php.exe file in each of these directories. If the file is found, it will be launched. If the file is not found, a corresponding message will be displayed.

In fact, only those who work a lot with the command line need to add something to the PATH variable. For example, you are a programmer and you place your programs in the C:\MyApps folder and you often run your command line utilities. In this case, you can add C:\MyApps to PATH and after that, to run programs from this folder, you no longer need to enter the full name each time (for example, C:\MyApps\parser.exe), but it will be enough in the command line enter only the filename:

parser.exe

Do I need to add PHP to environment variable on Windows

When installing and configuring PHP on Windows, it is not necessary to add the path to PHP to the PATH, but it is recommended to do so.

First, you can run PHP using the shorthand:

php my_script.php

instead

C:\Server\bin\PHP\php.exe my_script.php

Secondly, a number of extensions (which are included in the php.ini file) do not work correctly if you do not write the path to PHP in the PATH; this also applies to such a rather popular extension as cURL. In theory, this is some kind of bug of these extensions or PHP, but we ourselves will have to fix the situation, since this problem has existed for many years.

How to add PHP to system environment variables

So let's get started.

In the search box start typing “Edit the system environment variables

and open the corresponding settings window.

There click on “Environment Variables”, it will open:

In the “System variables” window, find and click on Path, then click “Edit”.

Then click the “New” button and enter “C:\Server\bin\PHP\” there:

Raise the entry to the very top:

Close all windows and save your changes.

Restart the server.

How to install Apache web server with PHP, MySQL and phpMyAdmin on Windows

Table of contents

1. Windows web server

2. How to install Apache on Windows

3. How to install PHP on Windows

4. PHP 8 setup

5. How to install MySQL on Windows

6. How to install phpMyAdmin on Windows

Conclusion


Windows web server

A web server is a program that is designed to process requests for websites and send website pages to users. The most popular example of a web server is Apache.

PHP is a programming language. Also, it called an environment for executing scripts written in PHP. In operating systems, including Windows, PHP can be installed independently, without a web server. In this case, programs (scripts) in PHP can be run from the command line. But web applications use PHP very often, this interpreter has become, in fact, the standard for web servers and therefore they are almost always installed together.

MySQL is a database management system (DBMS). It is also a standalone program, it is used to store data, search databases, modify and delete data. Web applications need persistent storage, so a DBMS is additionally installed for the web server. By the way, it is quite possible that you have heard about MariaDB - this is also a DBMS. MySQL came first, and then MariaDB forked from it. For web applications, both of these DBMS are interchangeable, that is, there is no difference. In this tutorial I will show the installation using MySQL as an example.

As for phpMyAdmin, this is just a PHP script that is designed to work with databases - it visually displays their contents, allows you to perform tasks in the graphical interface such as creating databases, creating tables, adding, changing and deleting information, etc. For this reason phpMyAdmin is quite popular, although it is not a required part of the web server.

The peculiarity of Apache and other web server components is that they have their roots in Linux. And these programs apply the basic concepts of this operating system in their work. For example, programs are very flexible in customization - you can install in any folder, sites can also be placed in any folder, including on another drive, not on the one where the web server itself is installed. Even the log files can be moved to the third disk and so on. The web server has many built-in modules - you can enable or disable them in any combination, you can add external modules. You can create many sites on the same web server and customize each site. But this flexible setting is done through text files - this is the approach (without a graphical interface) that allows you to describe any configuration

Do not be afraid of this - I will tell you what files need to be edited and what exactly to write in them.

We will not do any complex settings - our goal is just to install the web server on Windows. However, it would be strange not to use that kind of power in tuning at all. We will split the server into two directories: the first will contain executable files, and the second will contain data (website and database files). In the future, when the need arises to make backups of information or update the web server, you will understand how convenient this approach is!

We will install the server in a separate directory. To do this, create the “Server” directory in the root of the C:\ drive. Create 2 subdirectories in this directory: “bin” (for executable files) and “data” (for sites and databases).

Go to the “data” directory and create subfolders “DB” (for databases) and “htdocs” (for web sites) there.

Go to the “C:\Server\data\DB\” directory and create an empty “data” folder there.

For the operation of all components of the web server, the file “Visual C++ Redistributable for Visual Studio 2015-2019" is required - this is the official file from Microsoft. To download it follow the link. After downloading, run this file and complete the installation.

The preparatory steps are completed, we proceed to the installation of the web server components.

How to install Apache on Windows

Go to apachelounge.com/download and download the .zip archive with the web server:

Unpack the “Apache24” folder from this archive to “C:\Server\bin\”.

Go to the “C:\Server\bin\Apache24\conf\” directory and open the httpd.conf file with any text editor.

In it, we need to replace a number of lines.

Change

Define SRVROOT "c:/Apache24"

on

Define SRVROOT "c:/Server/bin/Apache24"

change

#ServerName www.example.com:80

on

ServerName localhost

change

DocumentRoot "${SRVROOT}/htdocs"
<Directory "${SRVROOT}/htdocs">

on

DocumentRoot "c:/Server/data/htdocs"
<Directory "c:/Server/data/htdocs">

change

DirectoryIndex index.html

on

DirectoryIndex index.php index.html index.htm

change

# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
#   AllowOverride FileInfo AuthConfig Limit
#
AllowOverride None

on

# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
#   AllowOverride FileInfo AuthConfig Limit
#
AllowOverride All

and change

#LoadModule rewrite_module modules/mod_rewrite.so

on

LoadModule rewrite_module modules/mod_rewrite.so

We save and close the file. That's it, Apache configuration is complete!

Open a command prompt (you can do this by pressing the Win+x keys at the same time). 

Select Windows PowerShell (Admin) there and copy there:

c:\Server\bin\Apache24\bin\httpd.exe -k install

Press Enter.

If you get a request from the firewall regarding Apache, then click Allow access.

Now we enter into the command line:

c:\Server\bin\Apache24\bin\httpd.exe -k start

And press Enter.

Now in the browser, type http://localhost/ and see the following:

This means the web server is running. To see the files there, add them to the c:\Server\data\htdocs\ directory - this is the main folder for the server data, where all sites will be located.

How to install PHP on Windows

Download PHP 8 from windows.php.net/download. Choose Thread Safe version, pay attention to select 64-bit computer architecture.

In the c:\Server\bin\ folder, create a “PHP” directory and copy the contents of the just downloaded archive into it.

In the file c:\Server\bin\Apache24\conf\httpd.conf at the very end we add the lines:

PHPIniDir "C:/Server/bin/PHP"
AddHandler application/x-httpd-php .php
LoadModule php_module "C:/Server/bin/php/php8apache2_4.dll"

And restart Apache:

c:\Server\bin\Apache24\bin\httpd.exe -k restart

In the c:\Server\data\htdocs\ directory, create a file called i.php, copy it to this file:

<?php
phpinfo ();

In your browser, open the link http://localhost/i.php. If you see something similar, then PHP is working:

PHP 8 setup

PHP is configured in the php.ini file. There is no php.ini in the zip archives intended for manual installation and for updates (this is done on purpose so that you do not accidentally delete your settings file during the update). But there are two others called php.ini-development and php.ini-production. Any of them, during manual installation, can be renamed to php.ini and further configured. On the localhost we will use php.ini-development.

Open the php.ini file with any text editor, looking for the line

;extension_dir = "ext"

and replace it with

extension_dir = "C:\Server\bin\PHP\ext\"

Now find the line group:

;extension=bz2
;extension=curl
;extension=ffi
;extension=ftp
;extension=fileinfo
;extension=gd
;extension=gettext
;extension=gmp
;extension=intl
;extension=imap
;extension=ldap
;extension=mbstring
;extension=exif      ; Must be after mbstring as it depends on it
;extension=mysqli
;extension=oci8_12c  ; Use with Oracle Database 12c Instant Client
;extension=oci8_19  ; Use with Oracle Database 19 Instant Client
;extension=odbc
;extension=openssl
;extension=pdo_firebird
;extension=pdo_mysql
;extension=pdo_oci
;extension=pdo_odbc
;extension=pdo_pgsql
;extension=pdo_sqlite
;extension=pgsql
;extension=shmop

and replace it with:

extension=bz2
extension=curl
extension=ffi
extension=ftp
extension=fileinfo
extension=gd
extension=gettext
extension=gmp
extension=intl
extension=imap
extension=ldap
extension=mbstring
extension=exif      ; Must be after mbstring as it depends on it
extension=mysqli
;extension=oci8_12c  ; Use with Oracle Database 12c Instant Client
extension=odbc
extension=openssl
;extension=pdo_firebird
extension=pdo_mysql
;extension=pdo_oci
extension=pdo_odbc
extension=pdo_pgsql
extension=pdo_sqlite
extension=pgsql
extension=shmop

now uncomment this line group:

;extension=soap
;extension=sockets
;extension=sodium
;extension=sqlite3
;extension=tidy
;extension=xsl 

it should look like this:

extension=soap
extension=sockets
extension=sodium
extension=sqlite3
extension=tidy
extension=xsl

With these actions, we enabled the extensions. They may be needed in different situations for different scripts. Save the file and restart Apache.

c:\Server\bin\Apache24\bin\httpd.exe -k restart

It is highly recommended to add the path to PHP to your “PATH” environment variable on Windows.

Related: How to add PHP path to %PATH% environment variable on Windows

How to install MySQL on Windows

The free version of MySQL is called MySQL Community Server. It can be downloaded from https://dev.mysql.com/downloads/mysql/. There is an executable installer on the same page, but I recommend downloading the ZIP archive.

On the download page, we are offered to register or log into an existing account - but this is optional. Just click on the link “No thanks, just start my download”.

Unpack the files from the just downloaded archive into the c:\Server\bin\ directory. The unpacked folder will be named approximately mysql-8.0.23-winx64 (depending on the version), rename it to mysql-8.0.

We go into this folder and create the my.ini file there. Now open this file with any text editor and add the following lines there:

[mysqld]
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
datadir="c:/Server/data/DB/data/"
default_authentication_plugin=mysql_native_password

Save and close it.

The configuration is complete, but you still need to perform the initialization and installation, for this we open the command line as administrator and sequentially enter there:

C:\Server\bin\mysql-8.0\bin\mysqld --initialize-insecure --user=root
C:\Server\bin\mysql-8.0\bin\mysqld --install
net start mysql

At the end of this process, the automatically generated files should appear in the C:\Server\data\DB\data\ directory.

The MySQL service will now start every time Windows starts.

How to install phpMyAdmin on Windows

The phpMyAdmin download site is phpmyadmin.net.

Direct link to the most recent version: phpMyAdmin-latest-all-languages.zip.

Copy the contents of the just downloaded archive to the c:\Server\data\htdocs\ directory. Rename this folder to “phpmyadmin”.

In the c:\Server\data\htdocs\phpmyadmin\ directory, create a config.inc.php file and copy there:

<?php

/* Servers configuration */
$i = 0;

/* Server: localhost [1] */
$i++;
$cfg['Servers'][$i]['verbose'] = '';
$cfg['Servers'][$i]['host'] = 'localhost';
$cfg['Servers'][$i]['port'] = '';
$cfg['Servers'][$i]['socket'] = '';
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['auth_type'] = 'cookie';
$cfg['Servers'][$i]['user'] = 'root';
$cfg['Servers'][$i]['password'] = '';
$cfg['Servers'][$i]['nopassword'] = true;
$cfg['Servers'][$i]['AllowNoPassword'] = true;

/* End of servers configuration */

$cfg['blowfish_secret'] = 'kjLGJ8g;Hj3mlHy+Gd~FE3mN{gIATs^1lX+T=KVYv{ubK*U0V';
$cfg['DefaultLang'] = 'en';
$cfg['ServerDefault'] = 1;
$cfg['UploadDir'] = '';
$cfg['SaveDir'] = '';

?>

In the browser, type http://localhost/phpmyadmin/

Enter root as the username. Leave the password field blank.

Conclusion

That's it - you now have your own personal local web server on your home computer.

If suddenly something didn't work out for you, then most likely you missed a step or did it incorrectly - try to do everything exactly according to the instructions. If the problem persists, then write about your error in the comments.

Pattaya in suspended animation

The Thai woman makes a video with commentary in English about how things are now in the resort areas of Thailand. In short, there are no people.

Some 7-Eleven and Family Mart even closed - I was surprised, I didn't think that most of the buyers are foreigners. Although appropriate, the money fed from tourists was gone. Even more - this is not the first “crisis”, during such periods they simply disperse to their villages and work and live there. When life is getting better, they return to the resort places.

Here she walks along Pattaya Beach Road from North Pattaya towards Central Pattaya.

5 star hotel redeveloped as a takeaway restaurant for locals (cheap). Some exchangers are closed. They bring sand to the sea, but the water carries it away. Expanded the pedestrian section of Beach Road. There are few people, a lot of places of entertainment are closed. It's not hot today, about +30℃. I'll walk from here to snack time.

Walking street Pattaya during the day - almost everything is closed, the government is trying to bury the electrical wires. She does not go from Central Pattaya, but from the opposite side of Walking street.

The creepiest thing is Walking street Pattaya at night. I would not recognize this place. Some kind of horror walk through Asian back streets.

She also has videos on her channel with fresh views of Thailand.

Wedding photography in Thailand (Pattaya) - examples

There are many beautiful and good girls in Thailand who you want to marry. (You can also marry bad girls - but then you don't have to complain crying).

My one of the most beautiful and favorite photos from Thailand are wedding photos. If you just want to see examples of salon photos of marriage, then scroll down a bit.

The services of a photographer can be ordered both for the ceremony itself and for photographing at any other time. If this is important for someone, it is not really necessary to get married.

Salon of wedding photography in Pattaya

There are many similar salons in Pattaya, I took my photos in this one:

It is located on Central Pattaya (Pattaya Clang), on the right side as you drive from Big C Extra towards Sukumvit Highway. Above, there is a link to the Google map - there you can see the street, as well as change the scale and switch to the map - if you wish, it's easy to find.

For a photo shoot, you need to bring your shoes and socks (they are not given), all other outfits, as well as makeup are provided free of charge (i.e. included in the cost of shooting).

The cost of wedding photos in Thailand

The cost is floating. It depends on:

1) how you bargain;

2) how many photos you will take.

My story is as follows: my wife from a shopping center brought a flyer with an advertisement for a wedding photo shoot promotion and said that she had already made a small advance payment. She told me that it is not expensive at all - in total, you will need to pay a thousand-something or two thousand-something baht.

After taking the photo, it turned out that yes, the price is small, but it only includes three photos from the whole set of photos taken. You can pick up more photos, but, of course, at a completely different price. The day after the shooting, we were invited to select the photos of interest to us.

There was no extra money, and indeed I had a feeling that I was being deceived. Therefore, I sent my wife with an indication that I will not pay a penny more than indicated in the flyer.

It is clear that a lot of photos were taken, they were all very beautiful and everything in the salon was done to ensure that the client pays for as many photos as possible. My wife called me a couple of times from the salon and told me the new price they asked for the photos - from 15-20 thousand baht the price was dropped to 5-7 thousand (I don't remember exactly). I flatly refused, but, just in case, I decided to call my father - he is also a professional photographer, but in Russia. He told me that for professional photography in Russia they would take more from me (even at the new currency rate), plus they would charge me separately for makeup and even costumes. In general, I called my wife, and we bought quite a lot of photos.

Also, the price included two large posters that need to be inserted into frames - it would have made great pictures and this is such an album (I don't know what it is called correctly):

My wife (unfortunately) left me. But the photographs, and with them the memory, remained. Let's see them already.

Examples of studio wedding photography in Thailand

How to set up Varnish, rule examples

Table of contents

1. How to change Varnish options

2. How to change cache retention time in Varnish

3. How to prevent Varnish from creating new cache for different browsers

4. A 403 page got into the Varnish cache and now it is shown to all users

5. How to make Apache logs show real IP address instead of 127.0.0.1 when used with Varnish

6. How to delete cookies that prevent caching

7. How to exclude certain pages from caching

8. How to exclude the home page from caching

9. How to increase the size of the Varnish cache

10. How to increase connection timeout in Varnish

11. How to exclude a specific site from Varnish caching

12. How to cache only a specific domain in Varnish

13. How to redirect HTTP to HTTPS in Varnish

14. How to delete cookies from all hosts except pages of a specific host

15. How to remove cookies from all pages except specific URLs

16. Adding and Removing HTTP Headers

17. Additional documentation


Once you've installed and configured Varnish to work with your web server. It would seem that everything is working fine and now you can move on to other things. But the truth is, with the default settings, Varnish is (almost) completely useless. The point is that by default:

  • cache storage time 2 minutes
  • a NEW cache is created for EACH User Agent. That is, if a page was requested by a user with a Chrome browser of one version, and then a user came with a different version of Chrome or with a different browser, then a new page will be created for him, instead of showing the one saved in the cache.

That is, the cache stores data for 2 minutes, which with a probability of 99% will not be shown to anyone for these two minutes, and then the data is deleted.

Moreover, with the default settings, Varnish is even harmful: a page with a response code of 503 (error on the server) or 403 (access denied) may be cached and this page will show everyone even when the problem is fixed.

In general, even though many tutorials on the Internet end up after installing Varnish and setting up a web server, you need to continue and do everything right. This is what this article is about. Here we will cover the minimum required configuration of Varnish to be useful and not harmful, and will also list examples of Varnish configuration that you can use in various situations.

How to change Varnish options

You can configure Varnish in the /etc/varnish/default.vcl configuration file, as well as by editing the command line launch options:

systemctl edit --full varnish

We will use both methods.

For the changes made in the default.vcl file to take effect, you must run the command:

systemctl reload varnish

This command will reload the configuration but keep the cache.

To make the changes made to the start command take effect, you must run:

systemctl restart varnish

This command will not save the cache - it will be cleared.

How to change cache retention time in Varnish

By default, Varnish keeps the cache for only 2 minutes - it is not enough for many situations.

The issue of storing the cache in Varnish is quite complex and can be devoted to a separate article. Here's a look at the basics.

First, there are three periods that Varnish keeps data in its cache:

  • TTL - Time To Live. This is the lifetime of the data. This is the period in which the data is stored in the cache and is considered fresh. “Fresh” - this means that when a request is received to show the page, the contents of the cache will be returned.
  • Grace. Grace period. This period comes after TTL. The data is still stored in the cache, and when a request is received for it, data from the cache is returned, at the same time a data update is started - a request is made to the web server.
  • Keep. Storage period. This is the period after TTL and Grace. The data is still stored in the cache, but is served to the user under certain conditions.

All these periods can be customized:

  • in the config file
  • in the command line
  • via HTTP headers

We will use the configuration file /etc/varnish/default.vcl. Open it:

vim /etc/varnish/default.vcl

The settings need to be made in the vcl_backend_response section. There are three options available, corresponding to each period

  • beresp.ttl
  • beresp.grace
  • beresp.keep

Example:

sub vcl_backend_response {

	# First we set the TTL value for most of the content that needs to be cached
	set beresp.ttl = 10m;
	set beresp.grace = 2h;

	# Now we can set specific TTLs based on the content to be cached
	# For VoD, we set a medium-long TTL and a long grace period, since VoD
	# content is not prone to change. This allows us to use this cache
	# for the most requested content

	if (beresp.url ~ "/vod") {
		set beresp.ttl = 30m;
		set beresp.grace = 24h;
	}

	# For live content we use a very low TTL and an even smaller grace period
	# since the live content is no longer *live* once it has been consumed
	
	if (beresp.url ~ "/url") {
		set beresp.ttl = 10s;
		set beresp.grace = 2s;
	}
	
	# We are increasing the *keep* duration for IMS

	if (bereq.http.If-Modified-Since) {
		set beresp.keep = 10m;
	}
}

As you can see from the example, suffixes are used:

  • ms - milliseconds
  • s - seconds
  • m - minutes
  • h - hours
  • d - days
  • w - weeks
  • y - years

Additional Information:

How to prevent Varnish from creating new cache for different browsers

If the web server sends the HTTP header

Vary: User-Agent

Varnish then creates new pages for each browser version. That is, different cache entries will be created for Chrome 85 and Chrome 86! Apache sends this HTTP header by default.

Let's demonstrate this.

First, let's make a request to the website using port 8080 (that is, bypassing the cache, we connect directly to Apache):

curl -I 'http://w-e-b.site:8080/?act=all-country-ip&city=Pattaya'

Pay attention to the line:

Vary: User-Agent,Accept-Encoding

There may be other options, for example:

Vary: User-Agent

Now we will run the command twice

time curl -s 'https://w-e-b.site/?act=all-country-ip&city=Pattaya' -A 'Chrome' > /dev/null

In it, we measure the time and can make sure that the second time the cache is created, the time it takes to get the page is much shorter. We also specified Chrome as the User Agent.

But if you change the User Agent value, then you can see from the time spent that the cache is being re-created:

time curl -s 'https://w-e-b.site/?act=all-country-ip&city=Pattaya' -A 'Firefox' > /dev/null

This behavior of the caching system usually doesn't make sense. Therefore, you need to make it so that Varnish uses the same cache for all User Agents.

There are several ways to do this. The easiest is to change the web server settings so that it does not send the “Vary: User-Agent” HTTP header.

The following shows how to change Apache settings so that it doesn't send “Vary: User-Agent”.

On Debian and derivatives:

a2enmod headers

Then add the line to the configuration file /etc/apache2/apache2.conf:

Header set Vary "Accept-Encoding"

Reboot the server:

systemctl restart apache2

On Arch Linux and derivatives:

Make sure to uncomment the following line in /etc/httpd/conf/httpd.conf:

LoadModule headers_module modules/mod_headers.so

Then add the line to the same file (/etc/httpd/conf/httpd.conf):

Header set Vary "Accept-Encoding"

Reboot the server for the changes to take effect:

systemctl restart httpd.service

We have rewritten the Vary HTTP header to only have Accept-Encoding.

This setup is definitely worth the effort! Varnish will now have one cache for all web browsers.

Additional Information:

A 403 page got into the Varnish cache and now it is shown to all users

It may happen that a user who is prohibited from viewing the site (banned by IP or by User Agent) opened a page and it got into the cache. As a result, now all users will be shown a message that access is denied instead of the normal page.

To fix this, add the following setting:

sub vcl_backend_response {	
	if (beresp.status == 403 || beresp.status == 404 || beresp.status >= 500)
	{
		set beresp.ttl = 3s;
	}
}

How to make Apache logs show real IP address instead of 127.0.0.1 when used with Varnish

Since Apache is now receiving requests from Varnish, now the web server logs always show 127.0.0.1 as client IP addresses.

This can be fixed as follows:

  • We tell Varnish to add the X-Forwarded-For HTTP header containing the user's real IP address when sending requests to the Apache server.
  • And in Apache, we enable the mod_remoteip module and specify in the settings to use the IP address from the X-Forwarded-For HTTP header as the user's real IP address

That is, you need to configure both Varnish and Apache. But the setup is very useful and well worth it!

Let's start by configuring Varnish, add to its config file:

sub vcl_recv {
    if (req.restarts == 0) {
        if (req.http.X-Forwarded-For) {
           set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
       } else {
        set req.http.X-Forwarded-For = client.ip;
       }
    }
}

The web server settings differ slightly depending on the distribution.

On Debian and derivatives:

a2enmod remoteip

Then add the line to the configuration file /etc/apache2/apache2.conf:

RemoteIPHeader x-forwarded-for

Reboot the web server:

systemctl restart apache2

On Arch Linux and derivatives:

Make sure to uncomment the following line in /etc/httpd/conf/httpd.conf:

LoadModule remoteip_module modules/mod_remoteip.so

Then add the line to the same file:

RemoteIPHeader x-forwarded-for

Reboot the server for the changes to take effect:

systemctl restart httpd.service

Additional Information:

Some tutorials advise you to modify the Apache log format - this will also work if done correctly, but it will take longer.

How to delete cookies that prevent caching

Requests that send cookies are not cached. If you have ad units or metrics (counters) on your site, then we can say that Varnish does not cache anything. Moreover, in the case of cookies for ad networks and counters, this is also absolutely useless data for the server.

To remove all cookies and allow data to be cached add:

sub vcl_recv {
	unset req.http.Cookie;
}

This setting will disrupt sites that rely on cookies. For example, if you have a WordPress site with registration for users, then cookies are required to identify users. In such cases, it is better to use another solution, such as Memcached.

How to exclude certain pages from caching

To exclude from caching all paths containing the string “/path/for/exclude/” anywhere in the URL:

sub vcl_recv {
	if (req.url ~ "^/path/for/exclude/") {
		return (pass);
	}
}

Note that the ~ (tilde) character means that the search is performed using a regular expression. And the ^ (caret) character means the beginning of a line.

If you want to exclude URLs in which “/path/for/exclude/” appears in any part of the URL, then remove the ^ character, for example:

sub vcl_recv {
	if (req.url ~ "/path/for/exclude/") {
		return (pass);
	}
}

If you want the search to be performed not by a regular expression, but by an exact match, then replace ~ with ==, for example:

sub vcl_recv {
	if (req.url == "/path/for/exclude/") {
		return (pass);
	}
}

You can use the if () {} construct several times or combine them into one. Symbol || means logical “OR”:

sub vcl_recv {
	if (req.url ~ "myip" || req.url ~ "proxy-checker" || req.url ~ "my-user-agent") {
		return (pass);
	}
}

That is, the previous setting will exclude from caching all pages with the string “myip” or the string “proxy-checker” or the string “my-user-agent” anywhere in the URL.

An example in which, for the host suip.biz, pages containing the string “/ru” in the URL are excluded from the cache:

sub vcl_recv {
	if (req.http.host == "suip.biz" && req.url == "/ru") {
		return (pass);
	}
}

How to exclude the home page from caching

The following example excludes the home page from the cache for the host suip.biz:

sub vcl_recv {
	if (req.http.host == "suip.biz" && req.url == "/") {
		return (pass);
	}
}

How to increase the size of the Varnish cache

The cache size settings must be changed in the command launch line, for this:

systemctl edit --full varnish

To resize the in-memory cache, edit the -s option. By the way, malloc means that the cache is stored in RAM. You can specify to store it in a file - if you need it, then refer to the documentation.

Specify the desired cache size, for example -s malloc,1400m.

Restart the varnish service for the changes to take effect:

systemctl restart varnish

How to increase connection timeout in Varnish

If the sending of data from Varnish to the client is not complete within 60 seconds, the connection is dropped. For regular sites this is fine, but for video portals or long-term services it may not be enough.

Varnish has quite a few timeout parameters, you can view their value with the command:

varnishadm param.show | grep timeout

Output example:

backend_idle_timeout          60.000 [seconds] (default)
between_bytes_timeout         60.000 [seconds] (default)
cli_timeout                   60.000 [seconds] (default)
connect_timeout               3.500 [seconds] (default)
first_byte_timeout            60.000 [seconds] (default)
idle_send_timeout             60.000 [seconds] (default)
pipe_timeout                  60.000 [seconds] (default)
send_timeout                  60.000 [seconds] (default)
thread_pool_timeout           300.000 [seconds] (default)
timeout_idle                  5.000 [seconds] (default)
timeout_linger                0.050 [seconds] (default)

The parameter we need, which is responsible for the maximum time for sending data during one connection, is called send_timeout.

To change it, you need to edit the command line

systemctl edit --full varnish

Add an option to the command like this:

-p send_timeout=SECONDS

For example, to set a connection timeout for 10 minutes:

-p send_timeout=600

Please note that you cannot specify the “s” suffix or any other, otherwise the program will not start. You only need to specify a number that means seconds for the connection timeout.

With this setting, we changed the value of the maximum time for sending data. But there is another timeout that sets the maximum connection duration. To edit it, open the /etc/varnish/default.vcl file and add the settings there:

backend default {
	.connect_timeout = 600s;
	.first_byte_timeout = 600s;
	.between_bytes_timeout = 600s;
}

In order not to edit the service start options, I tried to add the send_timeout option to the config file, but in this case, the service does not start due to an error. If you know what the matter is and how to do without adding an option in the command line, then write in the comments.

Restart the varnish service for the changes to take effect:

systemctl restart varnish

Additional Information:

How to exclude a specific site from Varnish caching

Exception from hashing softocracy.ru:

sub vcl_recv {
   if (req.http.host ~ "(www\.)?softocracy\.ru") {
     return(pass);
   }
}

If you do not want any site to be cached, then exclude it from caching entirely. In the example above, change softocracy.ru to the site's domain, which should not be cached.

How to cache only a specific domain in Varnish

For all other domains, pass must be returned:

sub vcl_recv {
	# if it's any domain other than example.com, then skip caching
	if (! req.http.host ~ "(www\.)?example\.com") {
		return (pass);
	}
	# otherwise switch to default behavior
}

pass tells Varnish not to look into its cache, it will always receive content from the destination web server.

How to redirect HTTP to HTTPS in Varnish

If you use Varnish, it is no longer possible to redirect to HTTPS using the web server methods, since Apache no longer works with HTTPS connections and does not even listen on port 443.

The following example will show you how to configure Varnish to redirect all requests to suip.biz from HTTP to HTTPS.

import std;

sub vcl_recv {
	# We ask Varnish to give 750 status for HTTP requests from external IP to port 80,
	# but not with SSL Termination Proxy (Hitch).
	if ((client.ip != "127.0.0.1" && std.port(server.ip) == 80) && (req.http.host ~ "^(?i)(www\.)?suip.biz")) {
		set req.http.x-redir = "https://" + req.http.host + req.url;
		return (synth(750, ""));
	}
}


sub vcl_synth {
	# Listen to 750 status from vcl_recv.
	if (resp.status == 750) {
		// Redirect to HTTPS with a 301 status.
		set resp.status = 301;
		set resp.http.Location = req.http.x-redir;
		return(deliver);
	}
}

In the previous configuration example, replace the “suip.biz” domain with your own.

How to delete cookies from all hosts except pages of a specific host

If you want cookies to be deleted for pages of all hosts, except for the pages of a specific host, then use the setting like this:

sub vcl_recv {
	if (! req.http.host ~ "(www\.)?HERE\.DOMAIN") {
		unset req.http.Cookie;
	}
}

Instead of “HERE\.DOMAIN” enter the domain on which you do not need to delete cookies. Note the escaping of the point.

An example of a setting that deletes cookies from all domains, except for the pages of the domains wxmaxima.ru and softocracy.ru:

sub vcl_recv {
	if (! req.http.host ~ "(www\.)?wxmaxima\.ru" && ! req.http.host ~ "(www\.)?softocracy\.ru") {
		unset req.http.Cookie;
	}
}

Attention:

1. Restart the varnish service for the changes to take effect:

systemctl restart varnish

2. In the config file use “unset req.http.Cookie;” only once! If you use it two or more times, the behavior will not be what you expect. Group all the conditions you need using logical AND and OR before using “unset req.http.Cookie;” only.

How to remove cookies from all pages except specific URLs

If you want to exclude only certain pages from deleting cookies, then use the setting like this:

sub vcl_recv {
	if (! req.url ~ "^/PATH/FOR/EXCLUDE/") {
		unset req.http.Cookie;
	}
}

An example of a setting that deletes cookies from all domains, except for the pages of the wxmaxima.ru domains and pages on any domains whose URL contains the string “phpmyadmin”:

sub vcl_recv {
	if (! req.http.host ~ "(www\.)?wxmaxima\.ru" && ! req.url ~ "^/phpmyadmin") {
		unset req.http.Cookie;
	}
}

Attention:

1. Restart the varnish service for the changes to take effect:

systemctl restart varnish

2. In the config file use “unset req.http.Cookie;” only once! If you use it two or more times, the behavior will not be what you expect. Group all the conditions you need using logical AND and OR before using “unset req.http.Cookie;” only.

Adding and Removing HTTP Headers

Varnish Cache gives you the ability to modify, add, and remove HTTP headers for the request and response object.

Request headers

The vcl_recv subroutine is called at the beginning of the request, and this is where we will change the request headers. We will add a hello header with the value world and remove the user agent header.

sub vcl_recv {
	...

	set req.http.hello = "world";
	unset req.http.user-agent;

	...
}

The req.http object is for accessing any request header and is readable only from vcl_recv and vcl_deliver.

Response headers

vcl_backend_response is called when Varnish Cache receives response headers from the upstream service. We will change the header cache control and set it to “public, max-age=600”, and remove the server header.

sub vcl_backend_response {
	...

	set beresp.http.cache-control = "public, max-age=600";
	unset beresp.http.server;

	...
}

The beresp.http object is for accessing any response header and is readable only from vcl_backend_response and vcl_deliver.

Additional documentation:

ONVIF client with command line interface

ONVIF is an open industry forum that provides and promotes standardized interfaces for effective interoperability of IP-based physical security products.

ONVIF protocol can be found on security cameras. Using ONVIF protocol, you can receive information from IP cameras and control them (set settings, rotate, etc.).

This article will talk about python-onvif - an ONVIF client implemented in Python. This program can be considered the successor to foscam-python-lib - Foscam Python Library for H.264 IP Cameras (FI9821W/P/HD816W/P).

This instruction will show working examples of using ONVIF on the command line.

To install python-onvif run:

sudo pip3 install --upgrade onvif_zeep

If you get an error that the pip3 command was not found, then you need to install the python3-pip package (for Debian based distributions) or python-pip (for Arch Linux based distributions).

You can find various examples of using python-onvif (in Python code, in scripts, in interactive mode, etc.) on the program home page: https://github.com/quatanium/python-onvif. The documentation is not informative, and due to the fact that you need to specify the wsdl directory, the path to which by default differs from that used in the examples, and also differs from distribution to distribution, all examples turn out to be inoperative.

Therefore, I propose to consider the use of python-onvif using an example of working code.

Create an extractor.py file with the following content:

import sys
from onvif import ONVIFCamera

# check if a user has been specified to connect. If not, use an empty string as the username
if len(sys.argv) < 4:
	user = ''
else:
	user = sys.argv[3]

# check if a password was specified for the connection. If not, use an empty string as a password
if len(sys.argv) < 5:
	password = ''
else:
	password = sys.argv[4] 		

# we create an object indicating the host, port, user, password and path to wsdl
mycam = ONVIFCamera(sys.argv[1], sys.argv[2], user, password, '/usr/lib/python3.9/site-packages/wsdl/')

# request and display information about the device
resp = mycam.devicemgmt.GetDeviceInformation()
print (str(resp))

# request and display information about network interfaces
resp = mycam.devicemgmt.GetNetworkInterfaces()
print (str(resp))

# request the address of the media stream
media_service = mycam.create_media_service()
profiles = media_service.GetProfiles()
token = profiles[0].token
mycam = media_service.create_type('GetStreamUri')
mycam.ProfileToken = token
mycam.StreamSetup = {'Stream': 'RTP-Unicast', 'Transport': {'Protocol': 'RTSP'}}
print(media_service.GetStreamUri(mycam))

Notice the line

/usr/local/lib/python3.9/site-packages/wsdl/

You need to replace it with your own value. This line is fine for Kali Linux. And on BlackArch/Arch Linux you need to use the following line:

/usr/lib/python3.9/site-packages/wsdl/

When the Python version changes, the line may change as well.

The path to it can be found by the following algorithm:

sudo updatedb # Update DB with file information
locate accesscontrol.wsdl # We are looking for a file that is in the desired directory

For example found:

/usr/local/lib/python3.9/site-packages/wsdl/accesscontrol.wsdl

We take the entire line, except for the file name, that is, /usr/local/lib/python3.9/site-packages/wsdl/.

Run like this:

python3 extractor.py HOST PORT [USER_NAME] [PASSWORD]

The script makes three separate requests and displays three groups of data: device information, network interfaces, and media stream.

USER_NAME and PASSWORD are optional - if they are absent, an attempt will be made to connect with empty credentials.

Launch example:

python3 extractor.py 118.39.210.69 80

Output example:

{
    'Manufacturer': 'BOSCH',
    'Model': 'AUTODOME IP starlight 7000 HD',
    'FirmwareVersion': '25500593',
    'SerialNumber': '044123455',
    'HardwareId': 'F0004D43'
}
[{
    'Enabled': True,
    'Info': {
        'Name': 'Network Interface 1',
        'HwAddress': '00-07-5f-8b-5d-2b',
        'MTU': 1514
    },
    'Link': {
        'AdminSettings': {
            'AutoNegotiation': True,
            'Speed': 100,
            'Duplex': 'Full'
        },
        'OperSettings': {
            'AutoNegotiation': True,
            'Speed': 100,
            'Duplex': 'Full'
        },
        'InterfaceType': 6
    },
    'IPv4': {
        'Enabled': True,
        'Config': {
            'Manual': [],
            'LinkLocal': None,
            'FromDHCP': {
                'Address': '118.39.210.69',
                'PrefixLength': 24
            },
            'DHCP': True,
            '_value_1': None,
            '_attr_1': None
        }
    },
    'IPv6': None,
    'Extension': None,
    'token': '1',
    '_attr_1': {
}
}]
{
    'Uri': 'rtsp://118.39.210.69/rtsp_tunnel?h26x=4&line=1&inst=1',
    'InvalidAfterConnect': False,
    'InvalidAfterReboot': True,
    'Timeout': datetime.timedelta(0),
    '_value_1': None,
    '_attr_1': None
}

Manufacturer, MAC address, video URI:

    'Manufacturer': 'BOSCH',
        'HwAddress': '00-07-5f-8b-5d-2b',
    'Uri': 'rtsp://118.39.210.69/rtsp_tunnel?h26x=4&line=1&inst=1',

Now that we know how to create an ONVIFCamera object and perform elementary actions with it, we can return to https://github.com/quatanium/python-onvif for more examples.

Also see:

Loading...
X