In the middle of pandemic, with all commerce’s closed and no access to better cameras or equipment, I demonstrated the plug-in architecture that I created and I added to my CTOP.py Open Source Python Monitoring tool in a global conference for all IT in ABK (Activision Blizzard King).
I was so proud.
For that I cloned ctop into a Raspberry Pi 4 with Ubuntu LTS and had that motherboard which is a Christmas Tree LED attached to the GPIO.
As the CPU load on the Raspberry was low, the LED’s were green, and a voice (recorded and provided by an Irish friend) was played “The System is Healthy”.
Then I added load to the CPU and the LED’s changed.
And I added more load to the CPU and the LED’s turned to Red and and human voice “CPU load is too high”.
Voice is only played after a change in the state and with a cool down of a minute, to prevent flapping situations to keep the program chatting like a parrot :)
I should have shaved myself, but you know, it was a savage pandemic.
Also a manager from Blizzard DM me and told me that the touch pad being emerged was due to the battery swallowing and that it could explode, so I requested a replacement. Then I explained to other colleagues with the same symptom, and to others with no problems so they are not at risk if the same happened to them.
WFH made things that would be quickly detected in the offices by others (like the touchpad emerging) go unnoticed.
if you are looking for source code, it is in the CTOP’s gitlab repository. However it’s advanced Python plugin architecture code.
If you just look for a sample of how to power on the LED’s in different colors, and the tricks for solving any problem you may encounter, then look at it here:
For all my friends and followers, I started to translate my radio space “El nou món digital” (in Catalan) to English “The New Digital World”. I cover Science, Technology, Entertainment and Video games.
I decided to create a Jenkins Pipeline to deploy any updates to this pages and I updated it a bit at least to provide the most common information searched.
Don’t expect anything fancy at Front End level. :)
Cloud
I created a video about how to provision a Ubuntu Droplet in Digital Ocean.
It’s just for beginners, or if you used other CSP’s and you wonder how Digital Ocean user interface is.
It is really easy, to be honest. Amazon AWS should learn from them.
I also created another about how to provision using User Data Cloud Init feature:
This version fixes few bugs, and adds better Unit Testing Code Coverage, and is integrated with jenkins (provides Jenkinsfile and a Dockerfile ready to automate the testing pipeline)
Sudo ku solver, Sudoku solver in Python, and Engineering solving problem approach
I’ve created this video explaining my experience writing a program to solve two impossible, very annoying Sudokus. :)
We have released World of Warcraft Dragonflight Alpha.
In the sync meetings I lead with Wow SRE and product Team I was informed that streaming would be open. Myself I was granted to stream over twitch, but so far I didn’t want to stream video games in my engineering channels. It’s different kind of audiences IMO. Let me know if you would like to get video game streams in my streaming channels.
Lich King
Humor
If you have been in a madness of Servers of a Cluster getting irresponsible and having to cold reboot them from remote hands iDracs or similar, you know why my friends sent me this image :D
More funny things happened like when I was installing a VirtualBox VM live, and the ZFS pool became irresponsible due hardware errors in one SATA Spinning drive.
Things from broadcasting live…
Some of the feedback I got from talented Engineers is that even if the original matter to talk about was interesting, seeing everything falling apart live due to unexpected hardware problems, and me troubleshooting live is being the best of the show… which I found very amusing.
RAB Radio the new digital world
I keep doing my radio space for Radio America Barcelona, once per week, addressed to the Catalan Community across the world and expats.
This radio program, streamed also via Twitch, is available in Catalan language only. RAB.
Open Source
carleslibs
I’ve been working in version 1.0.8 branch, and after a session of refactor on Twitch where I found a bug in MenuUtils class, I fixed it and released v. 1.0.8. You can see the video on the link.
Now I’m working on the branch v. 1.0.9.
ctop
I’ve been working in the branch 0.8.9.
My first Twitch broadcast was about adding Unit Testing to MemUtils class.
This week I decommissioned my last physical server in a Data Center.
It has been a long journey since I created my company to launch my own projects, and I started having my own infrastructure, back at 2000.
I was offering VPS at that time, with VMWare as Hypervisor.
This last Rack Server served me well for 21 years.
Now everything is Cloud, and is not viable to host and maintain servers unless this is your main occupation. Server’s motherboards die, hard drives die and they need to be replaced. Maintaining infrastructure it’s a full time job and you require somebody to do it. Also using fixed servers only prevents you from moving fast, locks a lot of money, and from spawning more compute capacity.
If you are curious this Rack Server is a Super Micro with Intel Xeon processor and SCSI drives.
Security
Firewall
I keep blocking thousands of IP Addresses every day.
When I see a pattern of an IP trying an attacks against the Server I look at the IP and if it’s from a hosting provider I just block the entire range.
I keep blocking any IP Address coming from Russia or Belarus since they invaded Ukraine.
My Health
I visited the hospital for a programmed following on my health.
The analysis are super good, and it’s super clear that I’ve improved radically. My discipline with the diet, taking the medicines and doing exercise regularly has been crucial.
My Doctor is confident that I’ll have a full recovery, but to do so I need to loss a lot of weight in a year or two.
So, I need to focus on my health and in doing exercise, being happy and avoid any kind of negative stress.
The cost of the travels and the medicines have put some stress into my economy, but I’m fortunate that I can handle it.
Entertainment / Life / Reflections
Star Wars and racism
I’m really enjoying new Start Wars series Obi Wan, and I’ve been profoundly shocked to read that there are fans being racist against the black characters.
zpool watch is a small Python program for Linux workstations with graphical environment and ZFS, that checks every 30 seconds if your OpenZFS pools are Ok.
If a pool is not healthy, it displays a message in a window using tk inter.
Basically allows you to skip checking from the terminal zpool status continuously or to having to customize the ZED service to send an email and having to figure out how to it can spawn a window alert to the graphical system or what to do if the session has not been initiated.
carleslibs
Since last News from the Blog I’ve released carleslibs v.1.06, v.1.0.5 and v.1.0.4.
v.1.0.6 adds a new class OsUtils to deal with mostly-Linux Os tasks, like knowing the userid, the username, if it’s root, the distribution name and kernel version.
It also adds:
DatetimeUtils.sleep(i_seconds)
In v.1.0.5 I’ve included a new method for getting the Datetime in Unix Epoc format as Integer and increased Code Coverage to 95% for ScreenUtils class.
v. 1.0.4 contains a minor update, a method in StringUtils to escape html from a string.
It uses the library html (part of Python core) so it was small work to do for me to create this method, and the Unit Test for it, but I wanted to use carleslibs in more projects and adding it as core functionality, makes the code of these projects I’m working on, much more clear.
Minor refactors and adding more Code Coverage (Unit Testing), and protection in the code for division per zero when seconds passed as int are 0. (this was not an actual error, but is worth protecting the code just in case for the future)
Currently in Master there is a stable version of 0.8.9 mainly fixing https://gitlab.com/carles.mateo/ctop/-/issues/51 which was not detecting when CTOP was running inside a Docker Container (reporting Unable to decode DMI).
My Books
Docker Combat Guide
Added 20 new pages with some tricks, like clearing the logs (1.6GB in my workstation), using some cool tools, using bind mounts and using Docker in Windows from command line without activating Docker Desktop or WSL.
One of my SATA 2TB 2.5″ 5,400 rpm drive got damaged and so was generating errors, so that was a fantastic opportunity to show how to detect and deal with the situation to replace it with a new SATA 2TB 3.5″ 7,200 rpm and fix the pool.
The company sent me the Stein, which is sent to the employees that serve for two years, with a recognition and a celebration called “The Circle of Honor”.
Books purchased
I bought this book as often I discover new ways, better, to explain the things to my students.
Sometimes I buy books for beginners, as I can get explained what I want to do super fast and some times they teach nice tricks that I didn’t know. I have huge Django books, and it took a lot to finish them.
A simpler book may only talk about how to install and work with it under a platform (Windows or Mac, as instance) but it is all that I require as the command to create projects are the same cross platform.
For example, you can get to install and to create a simple project with ORM, connected to the database, very quickly.
Software
So I just discovered that Zoom has an option to draw in the shared screen, like Slack has. It is called Annotate. It is super useful for my classes. :)
Also discovered the icons in the Chat. It seems that not all the video calls accept it.
Hardware
As Working From Home I needed an scanner, I looked in Amazon and all of them were costing more than €200.
I changed my strategy and I bought a All-In-One from HP, which costed me €68.
So I’ll have a scanner and a backup printer, which always comes handy.
The nightmare started after I tried to connect it with Ubuntu.
Ubuntu was not recognizing it. Checking the manuals they force to configure the printer from an Android/iPhone app or from their web page, my understanding is for windows only. In any case I would not install the proprietary drivers in my Linux system.
Annoyed, I installed the Android application, and it was requesting to get Location permissions to configure it. No way. There was not possible to configure the printer without giving GPS/Location permissions to the app, so I cancelled the process.
I grabbed a Windows 10 laptop and plugged the All-in-one through the USB. I ran the wizard to search for Scanners and Printers and was not unable to use my scanner, only to configure as a printer, so I was forced to install HP drivers.
Irritated I did, and they were suggesting to configure the printer so I can print from Internet or from the phone. Thanks HP, you’ll be the next SolarWinds big-security-hole. I said no way, and in order to use the Wifi I have to agree to open that security door which is that the printer would be connected to Internet permanently, sending and receiving information. I said no, I’ll use only via USB.
Even selecting that, in order to scan, the Software forces me to create an account.
Disappointing. HP is doing very big stupid mistakes. They used to be a good company.
Since they stopped doing the drivers in Barcelona years ago, their Software and solutions (not the hardware) went to hell.
I checked the reviews in the App Store and so many people gave them 1 star and have problems… what a shame the way they created this solution.
This is a great Open Source, multi-platform editor, so I wanted to support the creator.
Security
Attacks: looking for exploits
This is just a sample of a set of attacks to the blog in a 3 minutes interval.
Another one this morning:
Now all are blocked in the Firewall.
This is a non stop practice from spammers and pirates that has been going on for years.
It was almost three decades ago, when I was the Linux responsible of an ISP, and I was installing a brand new Linux system connected to a service called “infovia”, at the time when Internet was used with dial-up and modems, and in the interval of time of the installation, it got hacked. I had the Ethernet connected. So then already, this was happening.
The morning I was writing this, I blocked thousands of offending Ip Addresses.
Protection solutions
I recommend you to use CloudFlare, is a CDN/Cache/Accelerator with DoS protection and even in its Free version is really useful.
Fun/Games
So I come with a game kind of Quiz that you can play with your friends, family or work colleagues working from home (WFH).
The idea is that the master shares screen and sound in Zoom, and then the rest connect to jackbox.tv and enter the code displayed on the master’s screen on their own browser, and an interactive game is started.
It is recommended that the master has two monitors so they can also play.
The games are so fun as a phrase appearing and people having to complete with a lie. If your friends vote your phrase, believing is true, you get points. If you vote the true answer, you get points too.
Very funny and recommendable.
Stuff
<humor>Skynet sent another terminator to end me, but I terminated it. Its processor lays exhibited in my home now</humor>
I bought a laminator.
It has also a ruler and a trimmer to cut the paper.
It was only €39 and I’ve to say that I’m very happy with the results.
It takes around 5 minutes to be ready, it takes to get to the hot-enough temperature, and feeds the pages slowly, around 50 secs a DIN-A4, but the results are worth the time.
I’ve protected my medical receipts and other value documents and the work was perfect. No bubbles at all. No big deal if the plastic covers are introduced not 100% straight. Even if you pass again an already plasticized document, all is good.
Fun
Databases
One of my friends sent me this image.
It is old, but still it’s fun. So it assumes the cameras of the parking or speed cameras, will OCR the plate to build a query, and that the code is not well protected. So basically is exploiting a Sql Injection.
Anybody working on the systems side, and with databases, knows how annoying are those potential situations.
I made a Donation to The Document Foundation, which makes the OpenOffice.
I use OpenOffice suite for writing my books and other documents, so I think it’s fair to contribute with their operating costs.
I’ve installed a plugin to add Code Highlighting
It also allows me to add blocks of Code, like this:
if CodeHighlighting.b_is_installed == True:
return VisualImprovement.update_to(10), "It's easy to read"
else:
return VisualImprovement.get_still_the_same_difficult_to_read(), "The blog lives in the medieval age"
Or Inline Code like print(self.awareness) which is also great
I’ve improved a bit, visually, the blog
I modified a bit my template. The changes consist into adding an id attribute to the table for the Quick Selection of the articles, and modifying my template: the styles in the file css/blocks.css and modifying the version in functions.php to reflect the new timestamp.
I also made that when the mouse goes over a link it is displayed in blue, and the already visited in a slightly darker blue.
#articles_selection a:hover {color: #2222FF;}
In the images below you can see the before, the intermediate, and the final.
I’ve also added a button to hide or show the Quick Selection
If you have a WordPress and jQuery does not work for you, with error:
TypeError: $ is not a function
$(document).ready(function(){
This is because for compatibility reasons you have to do different in WordPress:
jQuery(document).ready(function($){
I created several videos of 5 minutes to learn Unit Testing in Python 3 with pytest
The new PSU arrived and I replaced it on Saturday 16th
After 5 days working nonstop, with no problems, it seems clear that the failing item was the expensive, 850W, Corsair PSU. Sometimes it happens that a new component comes defective, but I paid overprice expecting quality, and it seems that the PSU was defective. Since the beginning the computer powered off every few hours max, so I have to finally assume that effectively it was the PSU. Disappointed with Corsair.
Firewall. This month I’ve blocked around 2,000 visitors that were mainly bots searching for exploits
I review the logs several times every day.
Actually I’ve blocked many more Ip’s in the firewall, as when I identify a company source of bots, I block all their range (Imagine, as I block entire class C addresses, there are 256 Ips each class C /24). This has translated into 2,000 visitors less per month to the blog, that were offenders.
I added some rules / guidelines to the Leave a Reply section
I moderate all the comments to keep the blog an useful and healthy place.
And I don’t publish Spam, or Marketing messages.
Abusive comments are blocked. Competent Engineers and nice human beings share their points and doubts with data, with technical arguments, with education, in a respectful and polite way. People that cannot observe a minimum decoration are not welcome.
I’ve published a new book (is still work in progress at 25%): Python 3 exercises for beginnings. Albert Ballart and Sandra Soriano coauthored with me. Thanks!.
The company has provided me and the members of my Team with unlimited Cloud capacity for testing and developing. That’s very nice. I plan to run some nice workloads to asses the performance of several solutions.
I’ve had tons of assignments from the university and with the classes to my students I had no much time.
The Health Department and the universities (CIT and UCC) offered free PCR for the students. I can only say Kudos again for Ireland.
They also offered certain more vulnerable students to vaccinate with Pfizer. More Kudos.
I bought some cases for Raspberry Pi 4.
I’ve bought a Wi-fi Mesh TP-Link to repeat the Wifi signal to zones of the house it was arriving weak. I’m very satisfied with this signal repeater.
This little buddy also provides an Ethernet to connect to a laptop or to a switch.
So far I have only tested with a laptop and works great.
I also released CTOP TAG v.0.8.4 and fixed some bugs in upcoming CTOP v.0.8.5 and added a new feature to display the File Descriptors in use and the Max FDs set in the Kernel. This feature needed some work as I did calculating all the FDs per process but Kernel has another value that comes from /proc/sys/fs/file-nr so I finally implemented this last value, fixed a bug in conversion of units with decimals (KBs without the decimal part) and added Unit Testing to check this cases. I was satisfied to release v. 0.8.5.
I’ve released cmemgzip v. 0.4.1, with colors, and is now a package available to install download from PIP.
pip3 install cmemgzip
I keep teaching to learn to program in Python, to become a Junior BackEnd Developer. To groups and individuals. It is so beautiful and wonderful seeing people transform their minds, their lives and their confidence with the coding skills and being able to chase after better jobs, and so funny and beautiful when this happens in a group dynamic, with friendship, that I’m thinking about assembling a new Group of Students. Those classes are in English but I got some suggestions to do group classes in Spanish. Even if Catalan is speak by around 11 Million people in the world if I have enough requests I’ll gladly do it. if you’re interested reach me in LinkedIn.
I’ve made CTOP.py compatible with Python 2 in order to support legacy Servers.
I’ve updated my book Python Combat Guide as I regularly do. One of the things you’ll find is my script to upload packages to pypi.org very easily so you and the rest of the world can download with pip install package. I’ve added my script in a blog post too.
One person named True as surname gets locked in iCloud due to a bug in their coding, confusing the surname by a boolean property.
Can get your coders to free my last name from icloud jail? Been locked out for 6+ months because of an uncapitalized t in TRUE, my surname but also a computer command.
I always smile when I find bugs. Bugs are everywhere, but always make me smile when important companies with tons of resources make them.
Today I show this bug received in my email from Glassdoor. :)
You can see how in the email they assume that key will exist, and doesn’t, so a KEY NOT FOUND error is displayed. :)
It is funny, but remembers us the importance of doing our best trying to avoid bugs, working seriously with Methodology, implementing Unit Testing, Functional Testing, having also QA.. all the mechanisms we can, as they can be the last line of defense to avoid releasing bugs to Production.
In the past I taught Java and PHP, and I still help my mates with their Assembler Practices in their Masters, so I could not stop laughing with this:
I’ve recorded two live sessions of Refactoring and Unit Testing working on the project cmemgzip v.0.4. It is basically the exercise of Refactoring a code that is too big, and extracting sections to small methods, and then adding pytest Unit Testing code coverage. I explain the arrays I use for testing a battery of cases instead of few of them. Is an exercise of talking loud what I do normally, so you can understand many small details so subtle as the order of parameters or consistency. I use this material so my students, colleagues learning Unit Testing, and other people can learn and make their code more resilient and high quality.
I’ve implemented a plugin for my Open Source Software CTOP that allows to interact with LED through the Raspberry Pi GPIO.
Plugins architecture in CTOP is something I really like. I had a lot of fun creating it, and is super powerful. Basically I load Python plugins on demand, that are able to register the methods to be called, using hooks. The plugins receives an instance of CTOP itself, injected as dependency, so they have completely visibility over all the status of the machines.
I’ve been developed a new version of CTOP compatible with Python 2.7. I will be adding to master branch and releasing as part of tag 0.8 soon.
I’ve released cmemgzip v.0.3 (stable) and v.0.4 (ongoing) which is a Python 3 Open Source utility like gzip, with the difference that the files are loaded into memory, then compressed in memory, then the original file is deleted and the compressed data is written to a .gz file.
That means that you can use it on systems that have no space left on the disk, as long as you have memory.
Please note, is possible to compress files much bigger than actual size of the memory as the Block size to be compressed can be indicated with parameter -m. Resulting gz file are completely compatible with gzip/gunzip, zcat, etc…
I’ve made a donation to vokoScreen author €25 (around $30 USD).
This is the Software that I’ve been using recently in Linux, and is very useful, specially having several monitors, so I’m thanking the author with a donation.
I’ve been using this Software to record the classes for my students, so I find nice to share the love.
For windows time ago I bought a commercial Software and it doesn’t do more than vokoScreen, which is available for Windows too.
During covid I lowered the price of my books to the minimum allowed by LeanPub, to $5 USD, and I find it so nice when people donate more, that I’m happy to contribute to brilliant authors. I support authors since I started to get my first salary several decades ago.
I’m an author. I create Software and Books. So I think is normal, common sense and healthy that we, developers, value our work and support other authors. :)
Long time ago I wrote an article about zoning and NDS-4600. A colleague asked me for help, as he bought a second hand unit and it was doing tests. I wrote and explained everything, and added this information to my ZFS in Ubuntu book.
I was very happy to see that you keep buying my books. :)
