Author Archives: Carles Mateo

CTOP.py video on the Activision Blizzard King IT Demos 2021 February

I’ve good memories of this video.

In the middle of pandemic, with all commerce’s closed and no access to better cameras or equipment, I demonstrated the plug-in architecture that I created and I added to my CTOP.py Open Source Python Monitoring tool in a global conference for all IT in ABK (Activision Blizzard King).

I was so proud.

For that I cloned ctop into a Raspberry Pi 4 with Ubuntu LTS and had that motherboard which is a Christmas Tree LED attached to the GPIO.

As the CPU load on the Raspberry was low, the LED’s were green, and a voice (recorded and provided by an Irish friend) was played “The System is Healthy”.

Then I added load to the CPU and the LED’s changed.

And I added more load to the CPU and the LED’s turned to Red and and human voice “CPU load is too high”.

Voice is only played after a change in the state and with a cool down of a minute, to prevent flapping situations to keep the program chatting like a parrot :)

I should have shaved myself, but you know, it was a savage pandemic.

Also a manager from Blizzard DM me and told me that the touch pad being emerged was due to the battery swallowing and that it could explode, so I requested a replacement. Then I explained to other colleagues with the same symptom, and to others with no problems so they are not at risk if the same happened to them.

WFH made things that would be quickly detected in the offices by others (like the touchpad emerging) go unnoticed.

if you are looking for source code, it is in the CTOP’s gitlab repository. However it’s advanced Python plugin architecture code.

If you just look for a sample of how to power on the LED’s in different colors, and the tricks for solving any problem you may encounter, then look at it here:

https://blog.carlesmateo.com/2021/02/16/raspberry-pi-solving-the-problem-gpio-setupself-number-gpio-in-self-gpio_pull_upsself-_pull-runtimeerror-not-running-on-a-rpi-in-ubuntu-20-04lts/

RAB El nou món digital, edició d’estiu / Summer Edition 2022 – 2022-08-08 [CA|EN]

Aquest és el guió per al proper programa El nou món digital a Ràdio Amèrica Barcelona, que s’emet els Dilluns a les 14:30 Ireland Time / 15:30 Zona horària Catalunya / 06:30 Pacific Time.

Disclaimer: Treballo per a Activision Blizzard. Totes les opinions són meves i no representen cap companyia.


This is the excerpt of my radio program at Radio America Barcelona that airs on Mondays 14:30 Irish Time / 15:30 Catalonia Time / 06:30 Pacific Time.

Disclaimer: I work for Activision Blizzard. Opinions are my own. My opinions do not represent any company.

El programa de ràdio a RAB fa vacances durant el mes d’Agost, i tornarà a emetres el 5 de Setembre.

Nogensmenys continuaré pujant notícies per a que estigueu informats durant l’estiu. :)

Darrera actualització / Last Update: 2022-08-07

The RAB radio program will be on holidays during August, coming back the 5th of September. However I’ll keep posting news.

Entreteniment

  • He vist Lightyear, a Disney+ i és molt guai i està en Català.
  • Evangelion 3.0+1.0 s’estrenarà als cinemes en Català! :)
  • Estic provant Google Stadia més a fons. M’he inscrit per un mes a prova de manera gratuïta. Hi estic jugant amb Linux, i la qualitat dels gràfics decep una mica. Per començar sembla que no suporten 4K en Linux, i és com si tota la imatge que envien estigués comprida amb jpeg, és a dir, hi ha una pèrdua de qualitat notable respecte a ho bé que ho veig a la XBOX X. També maregen una mica els jocs en 3D, i clar, tinc el monitor molt a prop comparat amb la distància amb la que jugo amb la tele. Jugo amb el teclat i el ratolí, i és infinitament més precís que mirar d’apuntar amb el comandament de la XBOX així que els zombies han palmat. ;)
    • A resolució Full HD es veu millor
    • Aquí pots comprar jocs, per a usar-los. I què passa amb els meus jocs si els vull vendre o regalar-los a algú?. No puc. I si un dia tanquen la plataforma també he perdut els diners.
  • I’m trying Google Stadia more in deep. I’ve enrolled for free one month.

Notícies d’estiu / Summer News

  • A Liban un home agafa d’hostatges els treballadors d’un banc i amenaça amb matar-los si no li tornen part dels diners que el banc a congelat del seu dipòsit (al país els bancs han congelat els dipòsits) perque el seu pare necessita una operació imminent i morirà, i necessita els diners per a pagar l’hospital. Quan el banc li ha desbloquejat $30,000 dels $210,000 que te estalviats i el banc va bloquejar, i que necessitava per a la operació del seu pare, s’ha entregat pacíficament.

Seguretat

Ciència

Internet / Societat

Notícies

Gadgets

Humor

Programes anteriors / Previous Programs

Programa anterior: RAB El nou món digital, edició d’estiu / Summer Edition 2022 [CA|EN]

Tots els programes: RAB

Docker with Ubuntu with telnet server and Python code to access via telnet

Explanations building the Container and running the python code

Here you can see this Python code to connect via Telnet and executing a command in a Server:

File: telnet_demo.py

#!/usr/bin/env python3
import telnetlib

s_host = "localhost"
s_user = "telnet"
s_password = "telnet"

o_tn = telnetlib.Telnet(s_host)

o_tn.read_until(b"login: ")
o_tn.write(s_user.encode('ascii') + b"\n")

o_tn.read_until(b"Password: ")

o_tn.write(s_password.encode('ascii') + b"\n")

o_tn.write(b"hostname\n")
o_tn.write(b"uname -a\n")
o_tn.write(b"ls -hal /\n")
o_tn.write(b"exit\n")

print(o_tn.read_all().decode('ascii'))

File: Dockerfile

FROM ubuntu:20.04

MAINTAINER Carles Mateo

ARG DEBIAN_FRONTEND=noninteractive

# This will make sure printing in the Screen when running in detached mode
ENV PYTHONUNBUFFERED=1

RUN apt-get update -y && apt install -y sudo telnetd vim systemctl  && apt-get clean

RUN adduser -gecos --disabled-password --shell /bin/bash telnet

RUN echo "telnet:telnet" | chpasswd

EXPOSE 23

CMD systemctl start inetd; while [ true ]; do sleep 60; done

You can see that I use chpasswd command to change the password for the user telnet and set it to telnet. That deals with the complexity of setting the encrypted password.

File: build_docker.sh

#!/bin/bash

s_DOCKER_IMAGE_NAME="ubuntu_telnet"

echo "We will build the Docker Image and name it: ${s_DOCKER_IMAGE_NAME}"
echo "After, we will be able to run a Docker Container based on it."

printf "Removing old image %s\n" "${s_DOCKER_IMAGE_NAME}"
sudo docker rm "${s_DOCKER_IMAGE_NAME}"

printf "Creating Docker Image %s\n" "${s_DOCKER_IMAGE_NAME}"
sudo docker build -t ${s_DOCKER_IMAGE_NAME} .
# If you don't want to use cache this is your line
# sudo docker build -t ${s_DOCKER_IMAGE_NAME} . --no-cache

i_EXIT_CODE=$?
if [ $i_EXIT_CODE -ne 0 ]; then
    printf "Error. Exit code %s\n" ${i_EXIT_CODE}
    exit
fi

echo "Ready to run ${s_DOCKER_IMAGE_NAME} Docker Container"
echo "To run in type: sudo docker run -it -p 23:23 --name ${s_DOCKER_IMAGE_NAME} ${s_DOCKER_IMAGE_NAME}"

When you run sudo ./build_docker.sh the image will be built. Then run it with:

sudo docker run -it -p 23:23 --name ubuntu_telnet ubuntu_telnet

If you get an error indicating that the port is in use, then your computer has already a process listening on the port 23, use another.

You will be able to stop the Container by pressing CTRL + C

From another terminal run the Python program:

python3 ./telnet_demo.py

Validate IP Addresses and Networks with CIDR in Python

Python has a built-in package named ipaddress

You don’t need to install anything to use it.

This simple code shows how to use it

import ipaddress


def check_ip(s_ip_or_net):
    b_valid = True
    try:
        # The IP Addresses are expected to be passed without / even if it's /32 it would fail
        # If it uses / so, the CIDR notation, check it as a Network, even if it's /32
        if "/" in s_ip_or_net:
            o_net = ipaddress.ip_network(s_ip_or_net)
        else:
            o_ip = ipaddress.ip_address(s_ip_or_net)

    except ValueError:
        b_valid = False

    return b_valid


if __name__ == "__main__":
    a_ips = ["127.0.0.2.4",
             "127.0.0.0",
             "192.168.0.0",
             "192.168.0.1",
             "192.168.0.1 ",
             "192.168.0. 1",
             "192.168.0.1/32",
             "192.168.0.1 /32",
             "192.168.0.0/32",
             "192.0.2.0/255.255.255.0",
             "0.0.0.0/31",
             "0.0.0.0/32",
             "0.0.0.0/33",
             "1.2.3.4",
             "1.2.3.4/24",
             "1.2.3.0/24"]

    for s_ip in a_ips:
        b_success = check_ip(s_ip)
        if b_success is True:
            print(f"The IP Address or Network {s_ip} is valid")
        else:
            print(f"The IP Address or Network {s_ip} is not valid")

And the output is like this:

The IP Address or Network 127.0.0.2.4 is not valid
The IP Address or Network 127.0.0.0 is valid
The IP Address or Network 192.168.0.0 is valid
The IP Address or Network 192.168.0.1 is valid
The IP Address or Network 192.168.0.1  is not valid
The IP Address or Network 192.168.0. 1 is not valid
The IP Address or Network 192.168.0.1/32 is valid
The IP Address or Network 192.168.0.1 /32 is not valid
The IP Address or Network 192.168.0.0/32 is valid
The IP Address or Network 192.0.2.0/255.255.255.0 is valid
The IP Address or Network 0.0.0.0/31 is valid
The IP Address or Network 0.0.0.0/32 is valid
The IP Address or Network 0.0.0.0/33 is not valid
The IP Address or Network 1.2.3.4 is valid
The IP Address or Network 1.2.3.4/24 is not valid
The IP Address or Network 1.2.3.0/24 is valid

As you can read in the code comments, ipaddress.ip_address() will not validate an IP Address with the CIDR notation, even if it’s /32.

You should strip the /32 or use ipaddress.ip_network() instead.

As you can see 1.2.3.4/24 is returned as not valid.

You can pass the parameter strict=False and it will be returned as valid.

ipaddress.ip_network(s_ip_or_net, strict=False)

See where is the space used in your Android phone from Ubuntu Terminal

So, you may have your Android phone full and you don’t know where the space is.

You may have tried Apps for Android but none shows the information in the detail you would like. Linux to the rescue.

First of all you need a cable able to transfer Data. It is a cable that will be connected to your computer, normally with an USB 3.0 cable and to your smartphone, normally with USB-C.

Sometimes phone’s connectors are dirty and don’t allow a stable connection. Your connections should allow a stable connection, otherwise the connection will be interrupted in the middle.

Once you connect the Android smartphone to the computer, unlock the phone and authorize the data connection.

You’ll see that your computer recognizes the phone:

Open the terminal and enter this directory:

cd /run/user/1000/gvfs/

Here you will see your device and the name is very evident.

The usual is to have just one device listed, but if you had several Android devices attached you may want to query first, in order to identify it.

The Android devices use a protocol named Media Transfer Protocol (MTP) when connecting to the USB port, and that’s different on the typical way to access the USB port.

usb-devices  | grep "Manufacturer=" -B 3

Run this command to see all the devices connected to the USB.

You may see Manufacturer=SAMSUNG or Manufacturer=OnePlus etc…

The information returned will allow you to identify your device in /run/user/1000/gvfs/

You may get different type of outputs, but if you get: 

T:  Bus=02 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 13 Spd=480 MxCh= 0

your device can be accessed inside:

cd mtp\:host\=%5Busb%3A002%2C013%5D/

There you’ll see Card and Phone.

You can enter the Phone internal storage or the SD Card storage directory:

cd Phone

To see how the space is distributed nicely I recommend you to use the program ncdu if you don’t have you can install it with:

sudo apt install ncdu

Then run ncdu:

ncdu

It will calculate the space…

… and let you know, sorted from more to less, and will allow you to browse the sub-directories with the keyboard arrow keys and enter to get major clarity.

For example, in my case I saw 8.5 GB in the folder Movies on the phone, where I didn’t download any movie, so I checked.

I entered inside by pressing Enter:

So Instagram App was keeping a copy all the videos I uploaded, in total 8.5 GB of my Phone’s space and never releasing them!.

Example for the SD card, where the usage was as expected:

RAB El nou món digital, edició d’estiu / Summer Edition 2022 [CA|EN]

Aquest és el guió per al proper programa El nou món digital a Ràdio Amèrica Barcelona, que s’emet els Dilluns a les 14:30 Ireland Time / 15:30 Zona horària Catalunya / 06:30 Pacific Time.

Disclaimer: Treballo per a Activision Blizzard. Totes les opinions són meves i no representen cap companyia.


This is the excerpt of my radio program at Radio America Barcelona that airs on Mondays 14:30 Irish Time / 15:30 Catalonia Time / 06:30 Pacific Time.

Disclaimer: I work for Activision Blizzard. Opinions are my own. My opinions do not represent any company.

El programa de ràdio a RAB fa vacances durant el mes d’Agost, i tornarà a emetres el 5 de Setembre.

Nogensmenys continuaré pujant notícies per a que estigueu informats durant l’estiu. :)

Program en Català
Program in English

Darrera actualització / Last Update: 2022-08-01

The RAB radio program will be on holidays during August, coming back the 5th of September. However I’ll keep posting news.

Entreteniment

  • Més anuncis en Català: Dragon Ball Super Heroi s’estrenarà als cinemes el 2 de Setembre
  • Han anunciat el trailer de la nova de Black Panther
  • The new Black Panther movie has been announced
  • I el 3 d’Agost a Disney Plus arriba Lightyear!
  • Un company de feina va ser al Comic Con i m’ha passat una foto de totes les estrenes super xules que vén el 2022 i 2023. Gràcies Mason!
  • A work colleague was at Comic Con and has sent me a picture of the new super cool movies coming in 2022 and 2023. Thanks Mason!

Notícies d’estiu / Summer News

  • Rosalia va cantar al Juliol en Català a València. Ja que els esforços d’espanya per a erradicar la llengua catalana, sempre celebrem que la nostra llengua brilli arreu.
  • Rosalia sang in July in Catalan in Valencia. As the efforts from Spain to eradicate the Catalan language are huge, we always celebrate our language shining anywhere.
  • Una estudiant de belles arts d’UK va protestar duent a la graduació que volia un reemborsament, doncs durant el covid la formació que han rebut està molt per sota de la formació presencial de la resta dels anys, i el preu és el mateix

  • Una companyia de telecomunicacions al Canadà, Rogers, va tenir un error en els seus processos i va deixar sense telefonia milions de clients, fins i tot el servei d’emergències va estar inaccessible durant un dia. La notícia parla de que va ser un error de codi però llegint la notícia sembla clar que la companyia no va invertir suficient en garantitzar la qualitat dels seus processos. Culpar a un error de codi és com culpar el gat de tot, quan els executius han sigut incompetents en posar els mitjans a IT: prou temps i personal per a que els enginyers dissenyessin una solució segura per a la posada en producció dels canvis. Per a mi és un altre exemple de no tenir cura dels clients.
  • A communications company in Canada, Rogers, had an error in their processes and let down without phone service millions of clients. Even emergencies was not accessible for a day. The new on the article linked says that it was an error in the code but reading the article it seems clear that the company did no invest in assuring the quality of their processes. Blaming an error in the code is like blaming the cat for everything, when the executives were incompetent in providing the means to IT: enough time and staff so the engineers could design a safe solution to release the changes to production. For me is another example of not caring about the customers.
  • Amazon ha pujat el preu de la subscripció d’Amazon Prime
  • Amazon has increased the price of the Prime subscription

Seguretat

  • La meva amiga Esther m’envia una captura del seu mòbil d’una estafa en que li envien un SMS on s’intenten fer passar per caixabanc. El més divertit és que ella no te compte en aquell banc.
  • My friend Esther sends me an screenshot from a fraud attempt via SMS where they try to impersonate caixabanc (a bank). The most funny is that she doesn’t have account in that bank.
  • Security flaws in a popular GPS module could allow hackers to track vehicles and to stop the vehicle by cutting the fuel reserves. This Chinese GPS costs $20 USD
    • And I think about how important is to have professional Software Developers which good salaries, with good managers, working in a democratic country with transparent and fair laws. Buying products sold by company that mistreat employees, or pay they few, or in dictatorship countries poses a security risk for all the customers

Internet / Societat

  • M’agrada CoffeeZilla perque desenmascara estafadors, i ha fet que més d’un vagi a presó
  • I like CoffeeZilla because he discovers scammers, and has made go to prison more than one

Notícies

  • Uber se segueix cobrint de glòria. Ha estat discriminant passetgers amb discapacitats

Gadgets

  • Canon i Nikon abandonen la tecnologia DSLRS per a centrar-se en les càmeres mirrorless

Humor

Programes anteriors / Previous Programs

Programa anterior: RAB El nou món digital 2022-07-25 [CA|EN]

Tots els programes: RAB

RAB El nou món digital 2022-07-25 [CA|EN]

Aquest és el guió per al proper programa El nou món digital a Ràdio Amèrica Barcelona, que s’emet els Dilluns a les 14:30 Ireland Time / 15:30 Zona horària Catalunya / 06:30 Pacific Time.

Disclaimer: Treballo per a Activision Blizzard. Totes les opinions són meves i no representen cap companyia.


This is the excerpt of my radio program at Radio America Barcelona that airs on Mondays 14:30 Irish Time / 15:30 Catalonia Time / 06:30 Pacific Time.

Disclaimer: I work for Activision Blizzard. Opinions are my own. My opinions do not represent any company.

Aquesta és la pàgina del programa del proper 25 de Juliol de 2022.

El nou món digital a RAB – en Català (Catalan language video)

El programa a RAB fa vacances durant el mes d’Agost, i tornarà a emetres el 5 de Setembre, però jo continuaré penjant notícies al blog.

This is the page for today’s program, July 25th 2022. A video in English will be uploaded after. The program in RAB will be on holidays during August, coming back the 5th of September, but I’ll continue uploading messages in the blog.

Darrer programa de la temporada d’estiu

Avui farem un programa fresc i curt, ja que és el darrer i tenim poc temps.

Video jocs i entreteniment

Videojocs

Commander Turtle

  • Parlaré d’un video joc que he fet per a que els nens puguin aprendre a programar dibuixant i puguin compartir les seves creacions passant el codi a altres amics/pares.

Elden Ring

Sèries

  • El 10 d’Agost arriba a Disney+ I am Groot, la sèrie d’animació d’aquest simpàtic heroi de Guardians de la Galàxia.

Trucs

  • Per seguretat, usar una tapeta per a la càmera del portàtil, o tapar la camara web externa
  • Quan necessites posar accents a un email, i uses el teclat d’un ordinador que no en te, com a bon/a expat
    • Teclat virtual per a Linux, Mac, Windows
    • També et pots comprar un teclat físic o un d’enrotllable

Actualitat

Programes anteriors

Programa anterior: RAB El nou món digital 2022-07-18 [CA|EN]

Tots els programes: RAB

News from the blog 2022-07-22

Carles in the News

For all my friends and followers, I started to translate my radio space “El nou món digital” (in Catalan) to English “The New Digital World”. I cover Science, Technology, Entertainment and Video games.

You can see the first programs I translated here:

Catalan and English programs RAB

And the script and links mentioned here:

https://blog.carlesmateo.com/2022/06/20/rab-el-nou-mon-digital-2022-06-27-ca/

Social

I’ve created an Instagram fan page for me / for the blog.

It is open for everybody.

https://www.instagram.com/blog_carlesmateo/

Videos for learning how to code

Learning How to code Python Unit Tests with pytest from the scratch, for beginners:

I added it to my series of videos:

https://blog.carlesmateo.com/learn-python-3-unit-testing/

Video for learning how to use RabbitMQ with Python in a Docker Container

https://blog.carlesmateo.com/2022/07/20/creating-a-rabbitmq-docker-container-accessed-with-python-and-pika/

Site carlesmateo.com

I use mostly this site https://blog.carlesmateo.com to centralize everything, so I’ve kept http://carlesmateo.com as a simple landing page made with my old (from 2013) ultra fast PHP Framework Catalonia Framework.

I decided to create a Jenkins Pipeline to deploy any updates to this pages and I updated it a bit at least to provide the most common information searched.

Don’t expect anything fancy at Front End level. :)

Cloud

I created a video about how to provision a Ubuntu Droplet in Digital Ocean.

It’s just for beginners, or if you used other CSP’s and you wonder how Digital Ocean user interface is.

It is really easy, to be honest. Amazon AWS should learn from them.

I also created another about how to provision using User Data Cloud Init feature:

https://blog.carlesmateo.com/2022/06/25/how-to-deploy-a-digitalocean-droplet-instance-and-use-userdata/

Books

My Books

I have updated Docker Combat Guide to show how to work with different users in the Dockerfile and accessing an interactive terminal.

I have also added how to create a Jenkins containerized.

Books I bought > CI/CD

I recommend you these books:

I’ve created a video about how to deploy jenkins in Docker, following the official documentation, in 4 minutes.

Install jenkins on Docker in ubuntu in 4 minutes

And posted an article about solving the error load key invalid format when provisioning from Jenkins with your SSH .pem Key in a variable.

https://blog.carlesmateo.com/2022/07/05/solving-linux-load-key-ssh_yourserver-invalid-format-when-provisioning-from-jenkins/

Open Source from Carles

CTOP.py

I have released a new version of CTOP, ctop.py version 0.8.9.

This version fixes few bugs, and adds better Unit Testing Code Coverage, and is integrated with jenkins (provides Jenkinsfile and a Dockerfile ready to automate the testing pipeline)

Sudo ku solver, Sudoku solver in Python, and Engineering solving problem approach

I’ve created this video explaining my experience writing a program to solve two impossible, very annoying Sudokus. :)

https://blog.carlesmateo.com/2022/04/26/working-on-a-sudoku-solver-in-python-source-code/

Commander Turtle: a small program in Python so children can learn to code by drawing

Children can learn to code and share their scripts, which are comma separated, easily.

Commander Turtle

My life at Activision Blizzard

We have released World of Warcraft Dragonflight Alpha.

In the sync meetings I lead with Wow SRE and product Team I was informed that streaming would be open. Myself I was granted to stream over twitch, but so far I didn’t want to stream video games in my engineering channels. It’s different kind of audiences IMO. Let me know if you would like to get video game streams in my streaming channels.

Lich King

Humor

If you have been in a madness of Servers of a Cluster getting irresponsible and having to cold reboot them from remote hands iDracs or similar, you know why my friends sent me this image :D

Creating a RabbitMQ Docker Container accessed with Python and pika

In this video, that I streamed on Twitch, I demonstrate the code showed here.

I launch the Docker Container and operated it a bit, so you can get to learn few tricks.

I created the RabbitMQ Docker installation based on the official RabbitMQ installation instructions for Ubuntu/Debian:

https://www.rabbitmq.com/install-debian.html#apt-cloudsmith

One interesting aspect is that I cover how the messages are delivered as byte sequence. I show this by sending Unicode characters

Files in the project

Dockerfile

FROM ubuntu:20.04

MAINTAINER Carles Mateo

ARG DEBIAN_FRONTEND=noninteractive

# This will make sure printing in the Screen when running in dettached mode
ENV PYTHONUNBUFFERED=1

ARG PATH_RABBIT_INSTALL=/tmp/rabbit_install/

ARG PATH_RABBIT_APP_PYTHON=/opt/rabbit_python/

RUN mkdir $PATH_RABBIT_INSTALL

COPY cloudsmith.sh $PATH_RABBIT_INSTALL

RUN chmod +x ${PATH_RABBIT_INSTALL}cloudsmith.sh

RUN apt-get update -y && apt install -y sudo python3 python3-pip mc htop less strace zip gzip lynx && apt-get clean

RUN ${PATH_RABBIT_INSTALL}cloudsmith.sh

RUN service rabbitmq-server start

RUN mkdir $PATH_RABBIT_APP_PYTHON

COPY requirements.txt $PATH_RABBIT_APP_PYTHON

WORKDIR $PATH_RABBIT_APP_PYTHON

RUN pwd

RUN pip install -r requirements.txt

COPY *.py $PATH_RABBIT_APP_PYTHON

COPY loop_send_get_messages.sh $PATH_RABBIT_APP_PYTHON

RUN chmod +x loop_send_get_messages.sh

CMD ./loop_send_get_messages.sh

cloudsmith.sh

#!/usr/bin/sh
# From: https://www.rabbitmq.com/install-debian.html#apt-cloudsmith

sudo apt-get update -y && apt-get install curl gnupg apt-transport-https -y

## Team RabbitMQ's main signing key
curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null
## Cloudsmith: modern Erlang repository
curl -1sLf https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/gpg.E495BB49CC4BBE5B.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/io.cloudsmith.rabbitmq.E495BB49CC4BBE5B.gpg > /dev/null
## Cloudsmith: RabbitMQ repository
curl -1sLf https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/gpg.9F4587F226208342.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/io.cloudsmith.rabbitmq.9F4587F226208342.gpg > /dev/null

## Add apt repositories maintained by Team RabbitMQ
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
## Provides modern Erlang/OTP releases
##
deb [signed-by=/usr/share/keyrings/io.cloudsmith.rabbitmq.E495BB49CC4BBE5B.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/deb/ubuntu bionic main
deb-src [signed-by=/usr/share/keyrings/io.cloudsmith.rabbitmq.E495BB49CC4BBE5B.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/deb/ubuntu bionic main

## Provides RabbitMQ
##
deb [signed-by=/usr/share/keyrings/io.cloudsmith.rabbitmq.9F4587F226208342.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/deb/ubuntu bionic main
deb-src [signed-by=/usr/share/keyrings/io.cloudsmith.rabbitmq.9F4587F226208342.gpg] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/deb/ubuntu bionic main
EOF

## Update package indices
sudo apt-get update -y

## Install Erlang packages
sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

## Install rabbitmq-server and its dependencies
sudo apt-get install rabbitmq-server -y --fix-missing

build_docker.sh

#!/bin/bash

s_DOCKER_IMAGE_NAME="rabbitmq"

echo "We will build the Docker Image and name it: ${s_DOCKER_IMAGE_NAME}"
echo "After, we will be able to run a Docker Container based on it."

printf "Removing old image %s\n" "${s_DOCKER_IMAGE_NAME}"
sudo docker rm "${s_DOCKER_IMAGE_NAME}"

printf "Creating Docker Image %s\n" "${s_DOCKER_IMAGE_NAME}"
sudo docker build -t ${s_DOCKER_IMAGE_NAME} . --no-cache

i_EXIT_CODE=$?
if [ $i_EXIT_CODE -ne 0 ]; then
    printf "Error. Exit code %s\n" ${i_EXIT_CODE}
    exit
fi

echo "Ready to run ${s_DOCKER_IMAGE_NAME} Docker Container"
echo "To run in type: sudo docker run -it --name ${s_DOCKER_IMAGE_NAME} ${s_DOCKER_IMAGE_NAME}"
echo "or just use run_in_docker.sh"

requirements.txt

pika

loop_send_get_messages.sh

#!/bin/bash

echo "Starting RabbitMQ"
service rabbitmq-server start

echo "Launching consumer in background which will be listening and executing the callback function"
python3 rabbitmq_getfrom.py &

while true; do

    i_MESSAGES=$(( RANDOM % 10 ))

    echo "Sending $i_MESSAGES messages"
    for i_MESSAGE in $(seq 1 $i_MESSAGES); do
        python3 rabbitmq_sendto.py
    done

    echo "Sleeping 5 seconds"
    sleep 5

done

echo "Exiting loop"

rabbitmq_sendto.py

#!/usr/bin/env python3
import pika
import time

connection = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))

channel = connection.channel()

channel.queue_declare(queue="hello")

s_now = str(time.time())

s_message = "Hello World! " + s_now + " Testing Unicode: çÇ àá😀"
channel.basic_publish(exchange="", routing_key="hello", body=s_message)
print(" [x] Sent '" + s_message + "'")
connection.close()

rabbitmq_getfrom.py

#!/usr/bin/env python3
import pika


def callback(ch, method, properties, body):
    # print(f" [x] Received in channel: {ch} method: {method} properties: {properties} body: {body}")
    print(f" [x] Received body: {body}")


connection = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))

channel = connection.channel()

channel.queue_declare(queue="hello")

print(" [*] Waiting for messages. To exit press Ctrl+C")

# This will loop
channel.basic_consume(queue="hello", on_message_callback=callback)
channel.start_consuming()

print("Finishing consumer")

Video: Parse the Tables from a Website with Python pandas

A quick video, of 3 minutes, that shows you how it works.

If you don’t have pandas installed you’ll have to install it and lxml, otherwise you’ll get an error:

  File "/home/carles/Desktop/code/carles/blog.carlesmateo.com-source-code/venv/lib/python3.8/site-packages/pandas/io/html.py", line 872, in _parser_dispatch
    raise ImportError("lxml not found, please install it")
ImportError: lxml not found, please install it

You can install both from PyCharm or from command line with:

pip install pandas
pip install lxml

And here the source code:

import pandas as pd


if __name__ == "__main__":

    # Do not truncate the data when printing
    pd.set_option('display.max_colwidth', None)
    # Do not truncate due to length of all the columns
    pd.set_option('display.max_columns', None)
    pd.set_option('display.max_rows', None)
    pd.set_option('display.width', 2000)
    # pd.set_option('display.float_format', '{:20,.2f}'.format)

    o_pd_my_movies = pd.read_html("https://blog.carlesmateo.com/movies-i-saw/")
    print(len(o_pd_my_movies))

    print(o_pd_my_movies[0])