Tag Archives: CTOP.py

News from the Blog 2021-05-22

  • The company has provided me and the members of my Team with unlimited Cloud capacity for testing and developing. That’s very nice. I plan to run some nice workloads to asses the performance of several solutions.
  • I’ve had tons of assignments from the university and with the classes to my students I had no much time.
  • The Health Department and the universities (CIT and UCC) offered free PCR for the students. I can only say Kudos again for Ireland.
  • They also offered certain more vulnerable students to vaccinate with Pfizer. More Kudos.
  • I bought some cases for Raspberry Pi 4.
  • I’ve bought a Wi-fi Mesh TP-Link to repeat the Wifi signal to zones of the house it was arriving weak. I’m very satisfied with this signal repeater.

This little buddy also provides an Ethernet to connect to a laptop or to a switch.

So far I have only tested with a laptop and works great.

  • I’ve written one article about the GUI Software I use on Linux Desktops.
  • I wrote another article about cloning applications you’ve working in other Linux Desktops under Wine.
  • I also released CTOP TAG v.0.8.4 and fixed some bugs in upcoming CTOP v.0.8.5 and added a new feature to display the File Descriptors in use and the Max FDs set in the Kernel. This feature needed some work as I did calculating all the FDs per process but Kernel has another value that comes from /proc/sys/fs/file-nr so I finally implemented this last value, fixed a bug in conversion of units with decimals (KBs without the decimal part) and added Unit Testing to check this cases. I was satisfied to release v. 0.8.5.

News for the blog 2021-03-21

  • I’ve released cmemgzip v. 0.4.1, with colors, and is now a package available to install download from PIP.
pip3 install cmemgzip
  • I keep teaching to learn to program in Python, to become a Junior BackEnd Developer. To groups and individuals.
    It is so beautiful and wonderful seeing people transform their minds, their lives and their confidence with the coding skills and being able to chase after better jobs, and so funny and beautiful when this happens in a group dynamic, with friendship, that I’m thinking about assembling a new Group of Students.
    Those classes are in English but I got some suggestions to do group classes in Spanish. Even if Catalan is speak by around 11 Million people in the world if I have enough requests I’ll gladly do it.
    if you’re interested reach me in LinkedIn.
  • This new is hilarious:

https://www.engadget.com/icloud-true-last-name-lock-out-192224309.html

One person named True as surname gets locked in iCloud due to a bug in their coding, confusing the surname by a boolean property.

  • Bugs everywhere.

I always smile when I find bugs. Bugs are everywhere, but always make me smile when important companies with tons of resources make them.

Today I show this bug received in my email from Glassdoor. :)

You can see how in the email they assume that key will exist, and doesn’t, so a KEY NOT FOUND error is displayed. :)

It is funny, but remembers us the importance of doing our best trying to avoid bugs, working seriously with Methodology, implementing Unit Testing, Functional Testing, having also QA.. all the mechanisms we can, as they can be the last line of defense to avoid releasing bugs to Production.

  • In the past I taught Java and PHP, and I still help my mates with their Assembler Practices in their Masters, so I could not stop laughing with this:

News from the Blog 2021-03-04

  • I’ve recorded two live sessions of Refactoring and Unit Testing working on the project cmemgzip v.0.4.
    It is basically the exercise of Refactoring a code that is too big, and extracting sections to small methods, and then adding pytest Unit Testing code coverage.
    I explain the arrays I use for testing a battery of cases instead of few of them.
    Is an exercise of talking loud what I do normally, so you can understand many small details so subtle as the order of parameters or consistency.
    I use this material so my students, colleagues learning Unit Testing, and other people can learn and make their code more resilient and high quality.
  • I’ve implemented a plugin for my Open Source Software CTOP that allows to interact with LED through the Raspberry Pi GPIO.

Plugins architecture in CTOP is something I really like. I had a lot of fun creating it, and is super powerful. Basically I load Python plugins on demand, that are able to register the methods to be called, using hooks. The plugins receives an instance of CTOP itself, injected as dependency, so they have completely visibility over all the status of the machines.

  • I’ve been developed a new version of CTOP compatible with Python 2.7.
    I will be adding to master branch and releasing as part of tag 0.8 soon.
  • I’ve released cmemgzip v.0.3 (stable) and v.0.4 (ongoing) which is a Python 3 Open Source utility like gzip, with the difference that the files are loaded into memory, then compressed in memory, then the original file is deleted and the compressed data is written to a .gz file.

That means that you can use it on systems that have no space left on the disk, as long as you have memory.

Please note, is possible to compress files much bigger than actual size of the memory as the Block size to be compressed can be indicated with parameter -m. Resulting gz file are completely compatible with gzip/gunzip, zcat, etc…

  • I’ve made a donation to vokoScreen author €25 (around $30 USD).

This is the Software that I’ve been using recently in Linux, and is very useful, specially having several monitors, so I’m thanking the author with a donation.

I’ve been using this Software to record the classes for my students, so I find nice to share the love.

For windows time ago I bought a commercial Software and it doesn’t do more than vokoScreen, which is available for Windows too.

During covid I lowered the price of my books to the minimum allowed by LeanPub, to $5 USD, and I find it so nice when people donate more, that I’m happy to contribute to brilliant authors. I support authors since I started to get my first salary several decades ago.

I’m an author. I create Software and Books. So I think is normal, common sense and healthy that we, developers, value our work and support other authors. :)

  • Long time ago I wrote an article about zoning and NDS-4600. A colleague asked me for help, as he bought a second hand unit and it was doing tests. I wrote and explained everything, and added this information to my ZFS in Ubuntu book.
  • I was very happy to see that you keep buying my books. :)

Raspberry Pi: Solving the problem GPIO.setup(self.number, GPIO.IN, self.GPIO_PULL_UPS[self._pull]) RuntimeError: Not running on a RPi! in Ubuntu 20.04LTS

So you are trying to program the Raspberry expansion PINS in Python, for example for this 3D LED Christmas Tree, and you’re getting the error:

GPIO.setup(self.number, GPIO.IN, self.GPIO_PULL_UPS[self._pull])
RuntimeError: Not running on a RPi!

I’m running this on Ubuntu 20.04LTS with a Raspberry 4.

The first thing:

Make sure you have an official Raspberry Pi charger.

Or at least, make sure your USB charger provides enough intensity to power the Raspberry and the LEDs.

The LED power comes from the motherboard and if Raspberry Pi has not enough energy this is not going to work.

My colleague Michela had her tree not working because of the charger was not able to provide enough energy. When she ordered a new charger, it worked like a charm.

Install the base Software

In order to communicate with General Purpose Input Output ports (GPIO) you need to install this Software:

sudo apt install python3-pip python3-gpiozero
sudo pip3 install giozero

In order to run the 3D LED Christmas Tree code samples

sudo pip3 install colorzero
sudo pip3 install rpi.gpio --upgrade

Reboot

It may be not required in some cases.

Download the Source code

https://github.com/ThePiHut/rgbxmastree#rgbxmastree

Run the samples as root

I saw many people stuck, in the forums, because of that.

To work with the LEDs you need to run the samples as root.

Where I used it?

I used it in my Open Source monitor Software CTOP.py in order to show the plugins/extensions capability of it. :)

News from the blog 2021-02-10

  • I have updated CTOP.py so now it detects if is running in a Google GCP instance.

So the list of instances/type of virtualization detected is:

  • Amazon AWS
  • Google GCP
  • OpenStack
  • VirtualBox
  • Docker containers
  • LXC

I’m working in detecting Raspberry Pi, models running CTOP, and in enabling the plugins system so anybody can easily expand the functionality of ctop.py.

v.0.7.8
Commented annotations and type hinting, to make CTOP compatible with Python 3.5.0.
Added Available RAM.
Added Google GCP detection.
Inform if it doesn't have permissions to decode DMI.
Print the userId (numeric) and the User (string), like: 1000 carles or 0 root.
Logic for swappiness <= 10 Ok, >10 and <= 30 warning, >30 red (alert).
Reduced digits for swap to 2, to avoid confusions.
  • I have updated my book Python 3 Combat Guide, with another full cycle, step by step, to convert an ugly script that escapes to shell to a nice OOP code with Unit Testing, step by step.
  • I have updated my book ZFS for Ubuntu 20.04 LTS, adding how to create a pool and Datasets for home, sharing NFS for the Media Player.
  • If you like Star Wars and the Mandalorian, you may laugh has much as I did with this video:

As you see I’m writing more articles about Windows, Mac Os X, and proprietary Software. Some of my colleagues work in companies and use proprietary Software, so I’ll be writing more articles about those ecosystems. I spend more time now with colleagues working on all kind of projects, and with students that have other problems too, so I help them. However my main focus is Open Source, Architecture, Scaling, programming in Python and Java.

Install Windows Subsystem for Linux, WSL 2 on Windows 10 64 bit, with Ubuntu, solution to error WslRegisterDistribution failed with error: 0x80070057

You know I love Linux. I was compiling my own Kernels back in 1995, when it took more than 24 hours in a 386, and working on the first ISPs in Barcelona managing the Linux Systems.

For my computers I prefer Linux, no doubt about it, but many multinationals I worked for have Windows option only for the Laptops and Desktops.

During years I had to deal with sending files to Linux or Unix (HP UX, Sun Solaris…) to process them and getting back the result. Some sort of ETL and Map Reduce in the prehistory of personal computers, taking in count aspects like Networks speeds too, available space, splitting files for processing.

When I was working as Senior Project Manager in Winterthur Insurance, now Axa, I had to run a lot of ETL (Extract Transform Load) for considerably big files, or when I was project manager and later head of department in Volkswagen gedas or later helping Start ups like Privalia. I can tell you that Windows didn’t like you to open editors to work with 1GB text or CSV file, and doesn’t like it, even if your computer has 16GB of Memory, and even if they do the simplicity of Bash scripts and using pipes, grep, awk… is so powerful that is very convenient to have those files processed using Linux.

And honestly is a pain to send back and forth files to a UNIX System just for Data Crunch. And a VM will be slow and use memory, and you have enable some sort of sharing with it so it can access the Data. Not to talk if you need to split the data files in blocks to be processed in parallel by several computers.

There are many solutions, like using Virtual Machines, Docker, external Servers, etc…

WSL allows you to run Linux command line tools inside Windows.

https://wiki.ubuntu.com/WSL

Having WSL allows things to be done much more straightforward, processing the files in your local windows hard drives.

Please note: Maybe you have enough using GitBash.

Error installing: WslRegisterDistribution failed with error: 0x80070057

When I installed it I found this error and look for an answer online. I found no solutions and many people suffering from the same problem, so I decided to publish an article on how to make it work.

The instructions I detail are based on the documentation from Canonical https://wiki.ubuntu.com/WSL and from Microsoft https://docs.microsoft.com/en-us/windows/wsl/install-win10 and the final solution I found.

Microsoft use Powershell to activate the features disabled in Windows, I did the same with Command Line, which I found more convenient for most of the non extremely tech people.

You will need:

  • For x64 systems: Version 1903 or higher, with Build 18362 or higher.

You can check your version of windows opening a Terminal (CMD.exe) and typing:

winver
  • For ARM64 systems: Version 2004 or higher, with Build 19041 or higher.

I’m not covering installing WSL for ARM, only for Intel/AMD Desktop/Laptops with Windows 10.

If you’re unsure, you can open a Terminal (CMD.exe) and run:

systeminfo | find "System Type"

If is a x64 system it will return: x64-based PC

Launch CMD.exe as Administrator and type:

dism /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

Enable Virtual Machine Feature:

dism /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

At this point you have to Restart Windows to complete this part of the installation, otherwise next step won’t work.

Download the Linux Kernel Update Package from here:

https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi

Execute wsl_update_x64.msi and grant permissions for modifying the system.

Now it’s crucial that you reboot again. Even if you’re not asked for.

That’s the tricky part.

Then set the version 2 of WSL as default:

C:\WINDOWS\system32>wsl --set-default-version 2
 For information on key differences with WSL 2 please visit https://aka.ms/wsl2

Installing Ubuntu (or Kali, or Debian, or openSUSE…)

Open the Microsoft Store or use Microsoft Store link to go directly to the installation of Ubuntu.

You have also several options as indicated in https://wiki.ubuntu.com/WSL:

The recommended way to install Ubuntu on WSL is through the Microsoft Store.

The following Ubuntu releases are available as apps on the Microsoft Store:

  • Ubuntu 16.04 LTS (Xenial) is the first release available for WSL. It supports the x64 architecture only. (offline installer: x64)
  • Ubuntu 18.04 LTS (Bionic) is the second LTS release and the first one supporting ARM64 systems, too. (offline installers: x64, ARM64)
  • Ubuntu 20.04 LTS (Focal) is the current LTS release, supporting both x64 and ARM64 architecture.
  • Ubuntu (without the release version) always follows the recommended release, switching over to the next one when it gets the first point release. Right now it installs Ubuntu 20.04 LTS.

Each app creates a separate root file system in which Ubuntu shells are opened but app updates don’t change the root file system afterwards. Installing a different app in parallel creates a different root file system allowing you to have both Ubuntu LTS releases installed and running in case you need it for keeping compatibility with other external systems. You can also upgrade your Ubuntu 16.04 to 18.04 by running ‘do-release-upgrade’ and have three different systems running in parallel, separating production and sandboxes for experiments.

But if you prefer, instead of using the Windows Store, you can download the appx.

In the same page mentioned you can do it for several versions, I attach the link for Ubuntu 20.04 LTS: https://aka.ms/wslubuntu2004

Assuming you used the Windows Store, if you did not reboot and try now to execute it for the first time, or you go to the Command Line and write bash, or open Ubuntu from Windows menu, whatever method you use, you’ll get the abovementioned error.

If that happens to you, just reboot and when you open it will work and will start the install and ask for a user and password:

From here you’re able to update the system, execute the text commands available in Linux, access to the Windows drives, launch htop, git, Python3, apt, wget… copy and paste between windows and Linux terminal, share PATH…

And of course you can run CTOP.py

Take in count that the space reported in / partition is not real, and that you have a 4GB swap.

You can access your C:\ Windows files from:

/mnt/c/

News from the Blog 2020-07-19

I start this section to mention things that are worth.

  1. I read this article from Slack outage in May. It is really nice:

https://slack.engineering/a-terrible-horrible-no-good-very-bad-day-at-slack-dfe05b485f82

2. I read this interesting article about Netflix and Hexagonal Architecture:

https://netflixtechblog.com/ready-for-changes-with-hexagonal-architecture-b315ec967749

3. This new from Amazon is interesting for everybody in the sector:

https://www.livemint.com/companies/news/amazon-extends-work-from-home-policy-for-corporate-employees-until-2021-11594874878195.html

4. I updated CTOP.py to v.0.7.6 adding some command line switches to:

  • Limit the number of loops (like just getting the info and exiting, for scripting)
  • Work mode in black and white
  • Sleep time between retraces, to avoid flickering over SSH, and reduce bandwidth

5. I’ve been asked by an editorial to print my books in paper, for all the world. I’m thinking about it.

6. I’ve been asked to talk in a DevOps event.

7. I’ve created a new section in the blog, for adding the score of the videogames I played.