Author Archives: Carles Mateo

Front End (Beginners): Check if an AJAX request is made

I created this little video to show how to use the Web Developer Tools to check if jQuery AJAX request is being made.

This is to help beginners in their journey learning to program Front End.

Please be aware that the page I used makes their own calls for marketing purposes. I cleared several times the history to show what is the real AJAX request we are doing.

Note: The recording program did not capture correctly the mouse position and this video has no sound.

The page I shown is:

https://www.w3schools.com/jquery/tryit.asp?filename=tryjquery_ajax_ajax

And some more info about jQuery .ajax method:

https://www.w3schools.com/jquery/ajax_ajax.asp

Normally you can try your HTML and JavaScript from your browser, opening the HTML file. However there are Security restrictions when it applies to AJAX requests or trying to access a local file from the browser.

You’ll need to run your JavaScript from an HTML page server from a Web Server. You can install apache if you use Linux, or WAMP if you use Windows or MAMP (Mac Apache MySQL PHP) if you use Mac or if you know how to you can create a Docker Container.

When you’re programming and your code does not trigger, pay attention to errors:

Top right, red circle, shows one error

News from the blog 2022-03-22

Support to Ukraine

I’m Catalan. In 1936 the fascist military leaded by franco raised in arms against the elected government of the Spanish Republic. The Italian and nazi German fascist in power bombed the Catalan population. Hundreds of thousands of innocent citizens were assassinated and millions of Catalan and Spaniards had to exile. The sons of those that were ruling with the dictator have been insisting in naming it a “civil war”, but it was the military lead by a fascist, revolting against the legitimate Republic and ending a democracy.

The dictatorship lasted until 1975, when the dictator died in the bed. The effects of the repression never abandoned Catalonia, and nowadays in Catalonia people is still detained by the Spanish police for talking the Catalan language in front of them, and our Parliament decisions are cancelled by the Spanish courts, for example to force the exit of a President of Catalonia that they didn’t like, or to force the Catalan schools to teach 25% of the time in Spanish attacking the Catalan teaching system.

During WW2 millions of Jews were mass murdered, also people from all the nations were assassinated.

Russian population suffered a lot also fighting the nazis.

Now we have to see how Russia’s army is invading Ukraine and murdering innocent citizens.

That’s horrible.

I know Engineers from Ukraine. Those guys were doing great building wealthy based on knowledge and working well for companies across the world. Now these people are being killed or Engineers, amongst all the brave population, are arming themselves to fight the invasion. Shells destroy beautiful cities and population are starving, and young soldiers from both sides will never be seen again by their mothers.

I wrote a small article on how to identify and block in the Ubuntu firewall the Ip’s from Russia and Belarus until Russia leaves Ukraine.

Let music play in solidarity with Ukraine. First is a Catalan group. Second is a famous Irish band in this epic song dedicated to the brave International Brigades, volunteers that fought the fascism in Spain and in Catalonia trying to make a better world.

The Blog

I’ve updated the SSL Certificate. The previous one I bought was issued for two years, and I renewed as it was due to expire.

I wrote a short article about how to update the SSL Certificates for Apache 2 in Ubuntu 20.04.

Articles

I published a small Python script to show the local datetime and the Unix Epoch Time.

Open Source

carleslibs

On the 6th of January I released carleslibs v.1.0.7

https://pypi.org/project/carleslibs/1.0.7/

The new version contains these improvements:

  • Modified OsUtils.get_total_and_free_space_in_gib() to return float instead of Integer.
  • Added HashUtils class with md5 for unicode Strings.
    • Produces the same as md5sum Linux tool.
  • Created FileUtils.create_folders() which creates all the subfolders in the path deep.
  • Unit Testing:
    • Added test_get_inodes_in_use_and_free() to test_osutils.py
    • Added two tests more to test_osutils.py
    • Added test for version.py
    • Tests for HashUtils class.

My books

Python 3 Exercises for Beginners

I have updated the book, offering solution to exercises 11.1, 11.2 (simple and encapsulated in a function) and I’ve created exercise 11.3.

If you purchased the book before, you can download any update for free.

Amazon AWS

I got an offer by a super editorial to publish my book Automating and Provisioning with Amazon Python 3 SDK boto3.

Honestly, my ego was flattered. It is a lot of reputation.

Although in the past I got an offer from another monstrously big editorial to publish world wide my book Python 3 Combat Guide and I also rejected, and an offer from a digital learning platform to create an interactive course from this same book.

I’ve rejected it again this time.

If you are curious, this is what I answered to them:

Hi XXXX,

I'm well, thank you. I hope you are doing well too.

Thanks for taking the time to explain your conditions to me.

I feel flattered by your editorial thinking about me. I respect your brand, as I mentioned, as I own several of your titles.

However, I have to refuse your offer.

Is not the first time an editor has offered to publish one or more of my books. For all over the world, with much higher economic expectations.

I'll tell you why I love being at LeanPub:

1- I own the rights. All of them.
2- I can publish updates, and my readers get them for free. As I add new materials, the value is maximized for my readers.
3- I get 80% of the royalties.
4- If a reader is not happy, they can return the book for 60 days.
5- I can create vouchers and give a discount to certain readers, or give for free to people that are poor and are trying to get a career in Engineering.

The community of readers are very honest, and I only got 2 returns. One of them I think was from an editorial that purchased the book, evaluated it, and they contacted me to publish it, and after I rejected they applied for the refund.

I teach classes, and I charge 125 EUR per hour. I can make much more by my side than the one time payment you offer. The compensation for the video seems really obsolete.

Also, I could be using Amazon self publishing, which also brings bigger margins than you.

So many thanks for your offer. I thought about it because of the reputation, but I already have a reputation. I've thousands of visits to my tech blog, and because of the higher royalties, even if I sell less books through LeanPub it is much more rewarding.

Thanks again and have a lovely day and rest of the week.


Best,
Carles

The provisioning in Amazon AWS through their SDK is a book I’m particularly proud, as it empowers the developers so much. And I provide source code so they can go from zero to hero, in a moment. Amazon should provide a project sample as I do, not difficult to follow documentation.

Teaching / Mentoring

As I was requested, I’ve been offering advice and having virtual coffees with some people that recently started their journey to become Software Engineers and wanted some guidance and advice.

It has been great seeing people putting passion and studying hard to make a better future for themselves and for their families.

I’ll probably add to the blog more contents for beginners, although it will continue being a blog dedicated to extreme IT, and to super cool Engineering skills and troubleshooting.

For my regular students I have a discord space where we can talk and they can meet new friends studying or working in Engineering.

Free Resources

This github link provides many free books in multiple languages:

https://github.com/EbookFoundation/free-programming-books

Tricks

Zoom can zoom the view. So if they are sharing their screen, and font is too small, you can give a relax to your eyes by using Zoom’s zoom feature. It is located in View.

My health

After being in the hospital in December 2021, with risk for my life, and after my incredible recuperation, I’ve got the good news that I don’t need anymore 2 of the 3 medicines I was taking in a daily basis. It looks well through a completely recovery thanks to my discipline, doing sport every day several times, and the fantastic Catalan doctors that are supporting me so well.

Since they found what was failing in me, and after the emergency treatments I started to sleep really well. All night. That’s a privilege that I didn’t have for long long time.

Humor

Sad but true history. How many super talented Engineers have been hired and then they were given a shitty laptop/workstation super slow? That happened to me when I was hired by Volkswagen IT: gedas. I was creating projects for very big companies and I calculated that I was wasting 2 hours of my time compiling. The computer did not had enough RAM and was using swap.

JavaScript fun (or not)

Yes, this works like this.

You can try yourself:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<html>
<body>
<script>
console.log("11" + 1)
console.log("11" - 1)
</script>
</body>
</html>
<html> <body> <script> console.log("11" + 1) console.log("11" - 1) </script> </body> </html>
<html>
<body>
<script>
    console.log("11" + 1)
    console.log("11" - 1)
</script>
</body>
</html>

As you can see if you open the Browser Developer tools (in Linux and Windows press F12 key):

Firsts steps programming in Python 3 with PyCharm and Git

This is a very simple guide for beginners, to benefit from using this amazing Python IDE with Git.

You may think that it’s a bit difficult and why you should use it. The first reason I would tell is because it marks the errors, not just syntax errors also if you have typos or you forgot an import or you forgot to declare a variable, has autocomplete, will mark when you don’t follow standards, etc… If you want to be a professional, this is the tool that most Python professional developers and companies use.

Go to Jetbrains page and from there select Developer Tools and PyCharm.

You can use this url to go directly:

https://www.jetbrains.com/pycharm/

PyCharm Community Edition is completely free.

If you don’t have it, create an account in Gitlab. I recommend you to use Gitlab over Github.

Before continuing you’ll have to validate your account by clicking the email that Gitlab sent you.

Create a New project, by pressing the blue button with this label.

Choose Create blank project.

If you select Public everybody will be able to see your code, I recommend you to start with a Private project.

Let checked Initialize repository with a README as this will save you plenty of time and will allow you to start working immediately.

The project will be created for you.

Hit the Clone blue button and it will provide two methods to clone your git repository. You will copy the Clone with HTTPS url.

There are different ways to clone a new project.

Assuming you already have one opened go to Main Menu > Git > Clone

In the URL field paste the URL you copied from Gitlab, in my case:

https://gitlab.com/carles.mateo/carles-mateo-demo-project.git

And press the blue botton Clone.

If you don’t have Git installed, PyCharm will tell you in red, and will allow you to install it for you. It is a very convenient way to install it, specially if you use Windows.

In Ubuntu or Debian Linux installing git is so easy as typing from the Terminal:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
sudo apt install git
sudo apt install git
sudo apt install git

If it’s your first time PyCharm will ask you for your credentials in Gitlab, so your username or email, and your password.

As it is you own new project and it’s empty, you can trust it. Hit Trust Project.

A window with tips will appear, you can close it.

As you see your project is empty. Only has a the default README.md file created by Gitlab.

Creating new Python files

On the top left, over the name of your project folder press with the Right button of the mouse and select New > Python File

Enter the name of the file, I choose helloworld and press Enter.

You will be asked if you want to add automatically the file to Git. This refers to your local Git repository. Say Yes and you will see the file changing from color Red, which means it is not added to the repository to Green. Green means that is added, but has never been committed.

Just add this line to helloworld.py

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
print("Hello World")
print("Hello World")
print("Hello World")

Note: I made a typo and I wrote “helloworl.py” instead of “helloworld.py”, I will show how to fix this later.

Now, press over the file name over the top left, with the Right button of the mouse, and select Run helloworl (In your case will be Run helloword)

Note: To be able to run a Python program, you need to have Python installed.

Note2: Do not start the name of your file by test as PyCharm will consider that you are writing Unit Testing Code, and will not show the option to run the code, but to run with pytest (if you have pytest framework installed).

And you see on the bottom how it worked and printed “Hello World”.

To rename the file you’ll click over it with the Right button of the mouse and select Refactor > Rename

Type the correct name, in my case “helloworld.py” and for the Scope select: Current File.

PyCharm is so powerful that can search for your entire project to see if other files are using that file and will update the code to reflect the file name change. By indicating Scope: Current File we are telling that the change affects only to this file, and by not marking Search for references and not marking Search in comments and strings you are indicating to PyCharm that it should not search in code and comments to update them with the new file. This option is really useful when you get more confident with programming.

Perfect. Now you want to commit the file to the repository and Push.

The first time PyCharm will ask you how people will know you, that is, your visible name in the Git History of Commits, and your email. You can mark to use these properties locally. This is basically because Git needs to know how to identify you. Consider that Git was created to work with Teams, so everybody should know who made what. Add your friendly name, like “Carles Mateo” and your email. I recommend you to check to use this info for Git globally (all the projects in your computer).

When you commit you’re expected to provide a Commit Message. As Git is used by teams, to work on the same code, try to explain briefly what your changes consisted on.

If you press Commit, you will see how helloworld.py is no longer in color green, now is black. This means that the file is up to date respect our local Git.

Go to the Main Menu and select Git > Push.

You’ll see a message in the bottom indicating that your code has been pushed to the repository.

Now, you can refresh your browser, and you’ll see the changes in Gitlab:

PyCharm vs VSCode

I got asked what are the advantage from PyCharm respect VSCode.

I may refer to this excellent comparison:

While VSCode has some great support for Python coding with the ‘Python’ plugin by Microsoft, PyCharm is truly designed for Python development and it shows.

https://tangenttechnologies.ca/blog/pycharm-vs-vscode/

https://tangenttechnologies.ca/blog/pycharm-vs-vscode/

Renewing a SSL Certificate for Apache2 in Ubuntu 20.04

First you have to generate a new csr and key files.

It is not recommended to reuse your old CSR file.

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
openssl req -new -newkey rsa:2048 -nodes -keyout blog_carles_mateo_com_2022.key -out blog_carlesmateo_com_2022.csr
openssl req -new -newkey rsa:2048 -nodes -keyout blog_carles_mateo_com_2022.key -out blog_carlesmateo_com_2022.csr
openssl req -new -newkey rsa:2048 -nodes -keyout blog_carles_mateo_com_2022.key -out blog_carlesmateo_com_2022.csr

As you can see I used the name of the domain and the year for the new files to be generated to easily distinguish them.

When you’re asked for the password, in the additional fields, keep that password safe in case you need the Cert to be reissued to you.

You’ll need to submit the CSR file to your SSL provider. They will return you the CRT and the CA-BUNDLE files.

Edit your Apache config file for the SSL site.

For example:

/etc/apache2/sites-enabled/11-https-blog-carlesmateo-com.conf

Your conf file will look similar to this:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
<VirtualHost *:443>
ServerAdmin webmaster@yourdomain.cat
DocumentRoot /opt/sites/www/blog.carlesmateo.com
ServerName blog.carlesmateo.com
SSLEngine on
SSLCertificateFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.crt
SSLCertificateKeyFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.key
SSLCertificateChainFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.ca-bundle
...
<VirtualHost *:443> ServerAdmin webmaster@yourdomain.cat DocumentRoot /opt/sites/www/blog.carlesmateo.com ServerName blog.carlesmateo.com SSLEngine on SSLCertificateFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.crt SSLCertificateKeyFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.key SSLCertificateChainFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.ca-bundle ...
<VirtualHost *:443>
	ServerAdmin webmaster@yourdomain.cat

	DocumentRoot /opt/sites/www/blog.carlesmateo.com
	ServerName blog.carlesmateo.com
        SSLEngine on
        SSLCertificateFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.crt
        SSLCertificateKeyFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.key
        SSLCertificateChainFile /opt/sites/certs/2022/blog_carlesmateo_com_2022.ca-bundle
...

Before restarting Apache2, test the configuration for syntax errors with:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
apache2ctl -t
apache2ctl -t
apache2ctl -t

If all is good, restart your Web Server with:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
service apache2 restart
service apache2 restart
service apache2 restart

With a browser, verify that the information of the domain is right. I recommend you to check in Firefox and Chrome at least.

A simple Python script to get the date and time in Unix Epoch and in local time

The Unix Epoch is the time, in seconds, that has passed since 1970-01-01 00:00:00 UTC.

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
#!/usr/bin/env python3
#
# Date time methods
#
# Author: Carles Mateo
# Creation Date: 2019-11-20 17:23 IST
# Description: Class to return Date, datetime, Unix EPOCH timestamp
#
import datetime
import time
class DateTimeUtils:
def get_unix_epoch(self):
"""
Will return the EPOCH Time. For convenience is returned as String
:return:
"""
s_now_epoch = str(int(time.time()))
return s_now_epoch
def get_datetime(self, b_milliseconds=False):
"""
Return the datetime with miliseconds in format YYYY-MM-DD HH:MM:SS.xxxxx
or without milliseconds as YYYY-MM-DD HH:MM:SS"""
if b_milliseconds is True:
s_now = str(datetime.datetime.now())
else:
s_now = str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
return s_now
def main():
o_datetime = DateTimeUtils()
s_now_epoch = o_datetime.get_unix_epoch()
s_now_date = o_datetime.get_datetime()
print(s_now_epoch)
print(s_now_date)
main()
#!/usr/bin/env python3 # # Date time methods # # Author: Carles Mateo # Creation Date: 2019-11-20 17:23 IST # Description: Class to return Date, datetime, Unix EPOCH timestamp # import datetime import time class DateTimeUtils: def get_unix_epoch(self): """ Will return the EPOCH Time. For convenience is returned as String :return: """ s_now_epoch = str(int(time.time())) return s_now_epoch def get_datetime(self, b_milliseconds=False): """ Return the datetime with miliseconds in format YYYY-MM-DD HH:MM:SS.xxxxx or without milliseconds as YYYY-MM-DD HH:MM:SS""" if b_milliseconds is True: s_now = str(datetime.datetime.now()) else: s_now = str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")) return s_now def main(): o_datetime = DateTimeUtils() s_now_epoch = o_datetime.get_unix_epoch() s_now_date = o_datetime.get_datetime() print(s_now_epoch) print(s_now_date) main()
#!/usr/bin/env python3

#
# Date time methods
#
# Author: Carles Mateo
# Creation Date: 2019-11-20 17:23 IST
# Description: Class to return Date, datetime, Unix EPOCH timestamp
#

import datetime
import time


class DateTimeUtils:

    def get_unix_epoch(self):
        """
        Will return the EPOCH Time. For convenience is returned as String
        :return:
        """
        s_now_epoch = str(int(time.time()))

        return s_now_epoch

    def get_datetime(self, b_milliseconds=False):
        """
        Return the datetime with miliseconds in format YYYY-MM-DD HH:MM:SS.xxxxx
        or without milliseconds as YYYY-MM-DD HH:MM:SS"""
        if b_milliseconds is True:
            s_now = str(datetime.datetime.now())
        else:
            s_now = str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))

        return s_now


def main():
    o_datetime = DateTimeUtils()
    s_now_epoch = o_datetime.get_unix_epoch()
    s_now_date = o_datetime.get_datetime()

    print(s_now_epoch)
    print(s_now_date)


main()

You can also get the code from:

https://gitlab.com/carles.mateo/blog.carlesmateo.com-source-code/

News from the Blog 2022-02-22

My Open Source projects

zpool watch

zpool watch is a small Python program for Linux workstations with graphical environment and ZFS, that checks every 30 seconds if your OpenZFS pools are Ok.

If a pool is not healthy, it displays a message in a window using tk inter.

Basically allows you to skip checking from the terminal zpool status continuously or to having to customize the ZED service to send an email and having to figure out how to it can spawn a window alert to the graphical system or what to do if the session has not been initiated.

carleslibs

Since last News from the Blog I’ve released carleslibs v.1.06, v.1.0.5 and v.1.0.4.

v.1.0.6 adds a new class OsUtils to deal with mostly-Linux Os tasks, like knowing the userid, the username, if it’s root, the distribution name and kernel version.

It also adds:

DatetimeUtils.sleep(i_seconds)

In v.1.0.5 I’ve included a new method for getting the Datetime in Unix Epoc format as Integer and increased Code Coverage to 95% for ScreenUtils class.

v. 1.0.4 contains a minor update, a method in StringUtils to escape html from a string.

It uses the library html (part of Python core) so it was small work to do for me to create this method, and the Unit Test for it, but I wanted to use carleslibs in more projects and adding it as core functionality, makes the code of these projects I’m working on, much more clear.

I’m working in the future v.1.0.7.

CTOP.py

I released the stable version 0.8.8 and tagged it.

Minor refactors and adding more Code Coverage (Unit Testing), and protection in the code for division per zero when seconds passed as int are 0. (this was not an actual error, but is worth protecting the code just in case for the future)

Working on branch 0.8.9.

Currently in Master there is a stable version of 0.8.9 mainly fixing https://gitlab.com/carles.mateo/ctop/-/issues/51 which was not detecting when CTOP was running inside a Docker Container (reporting Unable to decode DMI).

My Books

Docker Combat Guide

Added 20 new pages with some tricks, like clearing the logs (1.6GB in my workstation), using some cool tools, using bind mounts and using Docker in Windows from command line without activating Docker Desktop or WSL.

https://leanpub.com/docker-combat-guide/

BTW if you work with Windows and you cannot use Docker Desktop due to the new license, in this article I explain how to use docker stand alone in Windows, without using WSL.

ZFS on Ubuntu

One of my SATA 2TB 2.5″ 5,400 rpm drive got damaged and so was generating errors, so that was a fantastic opportunity to show how to detect and deal with the situation to replace it with a new SATA 2TB 3.5″ 7,200 rpm and fix the pool.

So I updated my ZFS on Ubuntu 20.04 LTS book.

Python 3

I’ve updated Python 3 Exercises for Beginners and added a new example of how to parse the <title> tag from an HTML page, using Beautifulsoup package, to the repository of Python 3 Combat Guide book.

I also added three new exercises, and solved them.

My friend Michela is translating the book to Italian. Thanks! :)

If you already purchased any of my books, you can download the updates of them when I upload them to LeanPub.

Free courses

Code Challenges

One of my students sent me this platform, which is kinda hackerrank, but oriented to video games. To solve code challenges by programming video games.

He is having plenty of fun:

https://www.codingame.com/start

More Symfony, APIs

If you enjoyed the Free Videos about Symfony, there is more.

https://symfonycast.com/screencast/api-platform

It talks about a bundle for building APIs.

And this tutorial explains in detail how to work with Webpack Encore:

https://symfonycasts.com/screencast/webpack-encore

100 Days of Code: Python Bootcamp

A friend of mine, and colleague, Michela, is following this bootcamp and recommends it for people learning from ground 0.

https://udemy.com/course/100-days-of-code/

My work at Blizzard

The company sent me the Stein, which is sent to the employees that serve for two years, with a recognition and a celebration called “The Circle of Honor”.

Books purchased

I bought this book as often I discover new ways, better, to explain the things to my students.

Sometimes I buy books for beginners, as I can get explained what I want to do super fast and some times they teach nice tricks that I didn’t know. I have huge Django books, and it took a lot to finish them.

A simpler book may only talk about how to install and work with it under a platform (Windows or Mac, as instance) but it is all that I require as the command to create projects are the same cross platform.

For example, you can get to install and to create a simple project with ORM, connected to the database, very quickly.

Software

So I just discovered that Zoom has an option to draw in the shared screen, like Slack has. It is called Annotate. It is super useful for my classes. :)

Also discovered the icons in the Chat. It seems that not all the video calls accept it.

Hardware

As Working From Home I needed an scanner, I looked in Amazon and all of them were costing more than €200.

I changed my strategy and I bought a All-In-One from HP, which costed me €68.

So I’ll have a scanner and a backup printer, which always comes handy.

The nightmare started after I tried to connect it with Ubuntu.

Ubuntu was not recognizing it. Checking the manuals they force to configure the printer from an Android/iPhone app or from their web page, my understanding is for windows only. In any case I would not install the proprietary drivers in my Linux system.

Annoyed, I installed the Android application, and it was requesting to get Location permissions to configure it. No way. There was not possible to configure the printer without giving GPS/Location permissions to the app, so I cancelled the process.

I grabbed a Windows 10 laptop and plugged the All-in-one through the USB. I ran the wizard to search for Scanners and Printers and was not unable to use my scanner, only to configure as a printer, so I was forced to install HP drivers.

Irritated I did, and they were suggesting to configure the printer so I can print from Internet or from the phone. Thanks HP, you’ll be the next SolarWinds big-security-hole. I said no way, and in order to use the Wifi I have to agree to open that security door which is that the printer would be connected to Internet permanently, sending and receiving information. I said no, I’ll use only via USB.

Even selecting that, in order to scan, the Software forces me to create an account.

Disappointing. HP is doing very big stupid mistakes. They used to be a good company.

Since they stopped doing the drivers in Barcelona years ago, their Software and solutions (not the hardware) went to hell.

I checked the reviews in the App Store and so many people gave them 1 star and have problems… what a shame the way they created this solution.

Donations

I made a donation to OpenShot Video Editor.

This is a great Open Source, multi-platform editor, so I wanted to support the creator.

Security

Attacks: looking for exploits

This is just a sample of a set of attacks to the blog in a 3 minutes interval.

Another one this morning:

Now all are blocked in the Firewall.

This is a non stop practice from spammers and pirates that has been going on for years.

It was almost three decades ago, when I was the Linux responsible of an ISP, and I was installing a brand new Linux system connected to a service called “infovia”, at the time when Internet was used with dial-up and modems, and in the interval of time of the installation, it got hacked. I had the Ethernet connected. So then already, this was happening.

The morning I was writing this, I blocked thousands of offending Ip Addresses.

Protection solutions

I recommend you to use CloudFlare, is a CDN/Cache/Accelerator with DoS protection and even in its Free version is really useful.

Fun/Games

So I come with a game kind of Quiz that you can play with your friends, family or work colleagues working from home (WFH).

The idea is that the master shares screen and sound in Zoom, and then the rest connect to jackbox.tv and enter the code displayed on the master’s screen on their own browser, and an interactive game is started.

It is recommended that the master has two monitors so they can also play.

The games are so fun as a phrase appearing and people having to complete with a lie. If your friends vote your phrase, believing is true, you get points. If you vote the true answer, you get points too.

Very funny and recommendable.

Stuff

<humor>Skynet sent another terminator to end me, but I terminated it. Its processor lays exhibited in my home now</humor>

I bought a laminator.

It has also a ruler and a trimmer to cut the paper.

It was only €39 and I’ve to say that I’m very happy with the results.

It takes around 5 minutes to be ready, it takes to get to the hot-enough temperature, and feeds the pages slowly, around 50 secs a DIN-A4, but the results are worth the time.

I’ve protected my medical receipts and other value documents and the work was perfect. No bubbles at all. No big deal if the plastic covers are introduced not 100% straight. Even if you pass again an already plasticized document, all is good.

Fun

Databases

One of my friends sent me this image.

It is old, but still it’s fun. So it assumes the cameras of the parking or speed cameras, will OCR the plate to build a query, and that the code is not well protected. So basically is exploiting a Sql Injection.

Anybody working on the systems side, and with databases, knows how annoying are those potential situations.

Python and coding

One of my colleagues shared this :)

Using Docker in Windows 10 without Windows Desktop with Docker Engine and without WSL

I added this article to my Docker Combat Guide book.

The change of license of Docker Desktop for Windows has been a low punch, a dirty one.

Many big companies use Windows as for the laptops and workstations, we like it or not.

You can setup a Linux development computer or Virtual Machine, you may argue, but things are not as easier.

Big companies have Software licenses assigned to corporation machines, so you may not use your Pycharm license in a Linux VM.

You may no use Docker Desktop either, if your company did not license it.

And finally you may need to have access to internal resources, like Artifactory, or Servers where access is granted via ACL, so only you, from your Development machine can access it. So you have to be able to run Docker locally.

After Docker introduced this changed of license I was using VirtualBox with NAT attached to the VPN Virtual Ethernet, and I port forwarded to be able to SSH, deploy, test, etc… from outside to my Linux VM, and it was working for a while, until with the last VirtualBox update and some Windows updates where pushed to my Windows box and my VirtualBox VMs stopped booting most of the times and having random problems.

I configured a new Linux VM in a Development Server, and I opened Docker API so my Pycharm’s workstation was able to deploy there and I was able to test. But the Dev Ip’s do not have access to the same Test Servers I need my Python Automation projects to reach (and quickly I used 50 GB of space), so I tried WSL. I like Pycharm I didn’t want to switch to VStudio Code because of their good Docker extensions, in any case I could not run my code locally with venv cause some of the packages where not available for Windows, so I needed Linux to run the Unit Testing and see the Code Coverage, run the code, etc…

I tried Hyper-V, tried with NAT External, but it was incompatible with my VPN.

Note: WSL can be used, but I wanted to use Docker Engine, not docker in WSL.

Installing Docker Command line binaries

The first thing I checked was the Docker downloads page.

I found the stand alone binary.

https://docs.docker.com/engine/install/binaries/#install-server-and-client-binaries-on-windows

In order to install it:

  1. Download the zip file from the page, in my case docker-20.10.12.zip
  2. Open PowerShell as Administrator
  3. Run: Expand-Archive C:\Users\carlesmateo\Downloads\docker-20.10.12.zip  -DestinationPath $Env:ProgramFiles\DockerCLI
  4. Run: cd $Env:ProgramFiles\DockerCLI\docker
  5. Run: .\dockerd.exe –register-service
  6. Run: Start-Service docker
  7. Check that Docker lists the running Containers (no errors) with: docker ps
  8. Check that the Service is running with: Get-Service docker
    You should expect something like:
Status Name DisplayName
------ ---- -----------
Running docker Docker Engine

Attempt to pull an Image with: docker pull ubuntu or docker pull php

If it works, you’re done, but most probably you will get it starting and get this error:

Error response from daemon: unsupported os linux

or this other error:

no matching manifest for windows/amd64 10.0.19042 in the manifest list entries

Depending on your system you may need to do certain things:

Turn Windows features on or off

I would make sure that are enabled:

  • Containers
  • Hyper-V
  • Virtual Machine Platform
  • Windows Hypervisor Platform
As you see WSL is not enabled

Press OK, and restart your computer.

Try Again to docker pull ubuntu

Enable Experimental Mode

Edit this file to enable experimental, you can run from the PowerShell:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
notepad C:\ProgramData\Docker\config\daemon.json
notepad C:\ProgramData\Docker\config\daemon.json
notepad C:\ProgramData\Docker\config\daemon.json
Change experimental from false to: true

Save the file and restart the Service:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
Restart-Service docker
Restart-Service docker
Restart-Service docker

Check if it works

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
Get-Service docker
Get-Service docker
Get-Service docker
Status   Name               DisplayName
------   ----               -----------
Running  docker             Docker Engine

Try if now it works.

Switch Daemon

If it is not working, try running:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
cd "C:\Program Files\Docker\Docker\"
.\DockerCli.exe -SwitchDaemon
cd "C:\Program Files\Docker\Docker\" .\DockerCli.exe -SwitchDaemon
cd "C:\Program Files\Docker\Docker\"
.\DockerCli.exe -SwitchDaemon

Give it two minutes and try to pull an image.

If it is still not working reboot, and try again:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
cd "C:\Program Files\Docker\Docker\"
.\DockerCli.exe -SwitchDaemon
cd "C:\Program Files\Docker\Docker\" .\DockerCli.exe -SwitchDaemon
cd "C:\Program Files\Docker\Docker\"
.\DockerCli.exe -SwitchDaemon

After it is working

I recommend you to add the new stand alone docker to the path, so you can call it from the terminal at any moment.

Edit the variable PATH of your user profile (not System wide)

I recommend you to have it on top after Python.

A simple example to grab the title of a page using Python and beautifulsoup4

A really simple code I added to my Python 3 Exercises for Beginners book, to grab the title of a Web page.

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
from urllib import request
from bs4 import BeautifulSoup
s_url = "https://blog.carlesmateo.com/movies-i-saw/"
s_html = request.urlopen(s_url).read().decode('utf8')
o_soup = BeautifulSoup(s_html, 'html.parser')
o_title = o_soup.find('title')
print(o_title.string) # Prints the tag string content
# Another possible way
if o_soup.title is not None:
s_title = o_soup.title.string
else:
s_title = o_title.title
print(s_title)
from urllib import request from bs4 import BeautifulSoup s_url = "https://blog.carlesmateo.com/movies-i-saw/" s_html = request.urlopen(s_url).read().decode('utf8') o_soup = BeautifulSoup(s_html, 'html.parser') o_title = o_soup.find('title') print(o_title.string) # Prints the tag string content # Another possible way if o_soup.title is not None: s_title = o_soup.title.string else: s_title = o_title.title print(s_title)
from urllib import request
from bs4 import BeautifulSoup

s_url = "https://blog.carlesmateo.com/movies-i-saw/"
s_html = request.urlopen(s_url).read().decode('utf8')

o_soup = BeautifulSoup(s_html, 'html.parser')
o_title = o_soup.find('title')

print(o_title.string) # Prints the tag string content

# Another possible way
if o_soup.title is not None:
    s_title = o_soup.title.string
else:
    s_title = o_title.title
print(s_title)

I also included this code in the code repository for Python 3 Combat Guide book.

https://gitlab.com/carles.mateo/python_combat_guide/-/blob/master/src/html_parse_beautifulsoup4.py

News from the Blog 2022-01-22

News for the Blog

It has been 9 years since I created the blog, and some articles have old content that still get many visitors. To make sure they get the clear picture and not obsolete information, all the articles of the taxonomy POST, will include the Date and Time when the article was first published.

I also removed the annoying link “Leave a comment” on top. I think it influences some people to leave comments before actually having read the article.

It is still possible to add comments, but they are on the bottom of the page. I believe it makes more sense this way. This is the way.

Technically that involved modifying the files of my template:

  • functions.php
  • content.php

My Books

Automating and Provisioning to Amazon AWS using boto3 Amazon’s SDK for Python 3

I finished my book about Automating and Provisioning to Amazon AWS using boto3 Amazon’s SDK for Python 3.

It’s 128 pages in Full size DIN-A4 DRM-Free, and comes with a link to code samples of a real project CLI Menu based.

Docker Combat Guide

I have updated my book Docker Combat Guide and I added a completely new section, including source code, to work with Docker’s Python SDK.

I show the Docker SDK by showing the code of an actual CLI program I wrote in Python 3.

Here you can see a video that demonstrates how I launched a project with three Docker Containers via Docker compose. The Containers have Python, Flask webserver, and redis as bridge between the two Python Containers.

All the source code are downloadable with the book.

Watch at 1080p at Full Screen for better experience

My Classes

In January I resumed the coding classes. I have new students, and few spots free in my agenda, as some of my students graduated and others have been hired as Software Developers. I can not be more proud. :)

Free Training

Symfony is one of the most popular PHP Frameworks.

You can learn it with these free videos:

https://symfonycasts.com/screencast/symfony/

Software Licenses I Purchased

Before leaving 2021, I registered WinRAR for Windows.

WinRAR is a compressing Software that has been with us for 19 years.

I’m pretty sure I registered it in the past, but these holidays I was out only with two Window laptops and I had to do some work for the university and WinRAR came it handy, so I decided to register it again.

I create Software and Books, and I earn my life with this, so it makes a lot of sense to pay others for their good work crafting Software.

Books I Purchased

I bought this book and by the moment is really good. I wanted to buy some updated books as all my Linux books have some years already. Also I keep my skills sharp by reading reading reading.

Hardware I purchased

So I bought a cheap car power inverter.

The ones I saw in Amazon were €120+ and they were not very good rated, so I opted to buy a cheap one in the supermarket and keep it on the car just in case one day I need it. (My new Asus Zenbook laptop has 18 hours of autonomy and I don’t charge it for days, but you never know)

For those that don’t know, a power inverter allows you to get a 220V (120V in US) plug, from the connection of the lighter from your car. Also you can get the energy from an external car battery. This comes in really handy to charge the laptop, cameras, your drone… if you are in the nature and you don’t have any plug near.

I bought one years ago to power up Raspberry Pi’s when I was doing Research for a project I was studying to launch.

Fun

Many friends are using Starlink as a substitute of fiber for their rural homes, and they are super happy with it.

One of them send me a very fun article.

It is in Italian, but you can google translate it.

https://leganerd.com/2022/01/10/starlink-ha-un-piccolo-e-adorabile-problema-con-i-gatti/

Anyways you can get the idea of what’s going on in the picture :)

So tell me… so your speed with Starlink drops 80% in winter uh… aha…

Random news about Software

I tried the voice recognition in Slack huddle, and it works pretty well. Also Zoom has this feature and they are great. Specially when you are in a group call, or in a class.

My health

I was experimenting some problems, so I scheduled an appointment to get blood analysis and to be checked. Just in case.

TL;TR I could have died.

The doctors saw my analysis and sent me to the hospital urgently, where they found something that was going to be lethal. For hours they were checking me and doing several more analysis and tests to discard false positives, etc… and they precisely found the issue and provided urgent treatment and confirmed that I could have died at any moment.

Basically I dodged a bullet.

I was doing certain healthy things that helped me in a situation that could have been deadly or extremely dangerous to my health.

With the treatment and my strict discipline, I reverted the situation really quick and now I have more health and more energy than before. I feel rejuvenated.

I’m feeling lucky that with my work, the classes, the books I wrote, etc… I didn’t have to worry about the expensive medicines, the transport, etc… It was a bad moment, during Christmas, with so many people on holidays, pharmacies and GPs closed, so I had to spend more time looking for, traveling, and to pay more than it would had been strictly necessary. Despite all the time I used to my health, I managed to finish my university duties on time, and I didn’t miss my duties at work after the hospital, neither I had to cancel any programming classes or mentoring sessions. Nobody out of my closest circle knew what was going on, with the exception of my boss, which I kept informed in real time, just in case there was any problem, as I didn’t want to let down the company and have my duties at work to be unattended if something major happened.

I was not afraid to die. Unfortunately I’ve lost very significant people since I was a child. Relatives, very appreciated bosses and colleagues which I considered my friends, and great friends of different circles. Illnesses, accidents, and a friend of mine committed suicide years ago, and some of my partners attempted it (before we know each other). When you see people that are so good leaving, this brings a sadness that cannot be explained with words. I have had a tough life.

We have a limited time, and he have freedom to make choices. Some people choose to be miserable, to mistreat others, to lie, to cheat, to be unfaithful, to lack ethic and integrity. Those are their choices. Their wasted time will not come back.

Some of my friends are doctors. I admire them. They save lives and improve the quality of life of people with health problems.

I like being an Engineer cause I can create things, I can build instead of destroy, I can help to improve the world, and I can help users to have a good time and to avoid the frustration of services being down. I chose to do a positive work. So many times I’ve been offered much bigger salaries to do something I didn’t like, or by companies that I don’t admire, and I refused. Cause I wanted to make a better world. I know many people don’t think like that, and they only take take take. They are even unable to understand my choices, even to believe that I’m like that. But it’s enough that I know what I’m doing, and that it makes sense for me, and that I know that I’m doing well. And then, one day, you realize, that doing well, being fair and nice even if other people stabbed you in the back, you got to know fantastic people like you, and people that adore you and love to have you in their life, in their companies… So I’m really fortunate. To all the good-hearted people around, that give without expect anything in return, that try to make the world a better place, thank you.

Solving problems when updating to GNS3 last version, running with VirtualBox

Last Updated: 2022-01-19 12:05 Irish Time

So here I explain how to solve a problem that was happening to a friend.

He uses GNS3 for the university, and after installing the latest version, which in this case is 2.2.29, it stopped working.

He had it configured to use the local Server and VirtualBox in Windows 10.

The first thing to check and to fix is the Ip address for Host Only.

If you use Linux or Mac, only certain Ip ranges can be used, or you’ll have to edit a config file inside /etc/vbox

So the first thing is to set an Ip Address in VirtualBox VM that will make you worry free.

So start VirtualBox VM directly, and when the VM boots, use the text menu application to Configure to a valid Ip from the range defined for Host Only.

You can check this in VirtualBox in File > Host Network Manager

In my initial test I picket this Ip for the VM:

192.168.56.100

But using 192.168.56.100 can bring problems as the default DHCP Server is defined with this Ip, so I switched to:

192.168.56.10

Press CTRL + X to save and exit.

The VM will reboot automatically. Wait until it has booted and ping 192.168.56.10 from the Command Prompt.

Now, open a Windows Command Prompt or a Linux/Mac Terminal in you computer and ping the Ip:

You should also be able to see the web interface going to:

http://192.168.56.10

If it works then power off the VM, as we will start it automatically when running GNS3 main program (not from VirtualBox).

Now launch GNS3 program. Wait 30 seconds until it initializes and go to Edit > Preferences

Make sure you have the configuration like this:

Pay special attention to the Port for the GNS3 VM.

It seems like the main problem of my friend was that he was using a previous version, and he updated, and the settings from the previous version were kept. In his previous version he had configured the port 3080, but the new GNS 3 Server version 2.2.29 in the VM was using port 80, as you saw in my previous screenshots. So GNS3 was unable to connect to the VM.

After fixing this, restart GNS3, stop the VM if was not automatically stopped, and start GNS3 again.

After one minute approx connecting, you’ll see it working fine.