I’ve released cmemgzip v. 0.4.1, with colors, and is now a package available to install download from PIP.
pip3 install cmemgzip
I keep teaching to learn to program in Python, to become a Junior BackEnd Developer. To groups and individuals. It is so beautiful and wonderful seeing people transform their minds, their lives and their confidence with the coding skills and being able to chase after better jobs, and so funny and beautiful when this happens in a group dynamic, with friendship, that I’m thinking about assembling a new Group of Students. Those classes are in English but I got some suggestions to do group classes in Spanish. Even if Catalan is speak by around 11 Million people in the world if I have enough requests I’ll gladly do it. if you’re interested reach me in LinkedIn.
I’ve made CTOP.py compatible with Python 2 in order to support legacy Servers.
I’ve updated my book Python Combat Guide as I regularly do. One of the things you’ll find is my script to upload packages to pypi.org very easily so you and the rest of the world can download with pip install package. I’ve added my script in a blog post too.
One person named True as surname gets locked in iCloud due to a bug in their coding, confusing the surname by a boolean property.
Can get your coders to free my last name from icloud jail? Been locked out for 6+ months because of an uncapitalized t in TRUE, my surname but also a computer command.
I always smile when I find bugs. Bugs are everywhere, but always make me smile when important companies with tons of resources make them.
Today I show this bug received in my email from Glassdoor. :)
You can see how in the email they assume that key will exist, and doesn’t, so a KEY NOT FOUND error is displayed. :)
It is funny, but remembers us the importance of doing our best trying to avoid bugs, working seriously with Methodology, implementing Unit Testing, Functional Testing, having also QA.. all the mechanisms we can, as they can be the last line of defense to avoid releasing bugs to Production.
In the past I taught Java and PHP, and I still help my mates with their Assembler Practices in their Masters, so I could not stop laughing with this:
I’ve recorded two live sessions of Refactoring and Unit Testing working on the project cmemgzip v.0.4. It is basically the exercise of Refactoring a code that is too big, and extracting sections to small methods, and then adding pytest Unit Testing code coverage. I explain the arrays I use for testing a battery of cases instead of few of them. Is an exercise of talking loud what I do normally, so you can understand many small details so subtle as the order of parameters or consistency. I use this material so my students, colleagues learning Unit Testing, and other people can learn and make their code more resilient and high quality.
I’ve implemented a plugin for my Open Source Software CTOP that allows to interact with LED through the Raspberry Pi GPIO.
Plugins architecture in CTOP is something I really like. I had a lot of fun creating it, and is super powerful. Basically I load Python plugins on demand, that are able to register the methods to be called, using hooks. The plugins receives an instance of CTOP itself, injected as dependency, so they have completely visibility over all the status of the machines.
I’ve been developed a new version of CTOP compatible with Python 2.7. I will be adding to master branch and releasing as part of tag 0.8 soon.
I’ve released cmemgzip v.0.3 (stable) and v.0.4 (ongoing) which is a Python 3 Open Source utility like gzip, with the difference that the files are loaded into memory, then compressed in memory, then the original file is deleted and the compressed data is written to a .gz file.
That means that you can use it on systems that have no space left on the disk, as long as you have memory.
Please note, is possible to compress files much bigger than actual size of the memory as the Block size to be compressed can be indicated with parameter -m. Resulting gz file are completely compatible with gzip/gunzip, zcat, etc…
I’ve made a donation to vokoScreen author €25 (around $30 USD).
This is the Software that I’ve been using recently in Linux, and is very useful, specially having several monitors, so I’m thanking the author with a donation.
I’ve been using this Software to record the classes for my students, so I find nice to share the love.
For windows time ago I bought a commercial Software and it doesn’t do more than vokoScreen, which is available for Windows too.
During covid I lowered the price of my books to the minimum allowed by LeanPub, to $5 USD, and I find it so nice when people donate more, that I’m happy to contribute to brilliant authors. I support authors since I started to get my first salary several decades ago.
I’m an author. I create Software and Books. So I think is normal, common sense and healthy that we, developers, value our work and support other authors. :)
Long time ago I wrote an article about zoning and NDS-4600. A colleague asked me for help, as he bought a second hand unit and it was doing tests. I wrote and explained everything, and added this information to my ZFS in Ubuntu book.
I was very happy to see that you keep buying my books. :)
I have updated CTOP.py so now it detects if is running in a Google GCP instance.
So the list of instances/type of virtualization detected is:
Amazon AWS
Google GCP
OpenStack
VirtualBox
Docker containers
LXC
I’m working in detecting Raspberry Pi, models running CTOP, and in enabling the plugins system so anybody can easily expand the functionality of ctop.py.
v.0.7.8 Commented annotations and type hinting, to make CTOP compatible with Python 3.5.0. Added Available RAM. Added Google GCP detection. Inform if it doesn't have permissions to decode DMI. Print the userId (numeric) and the User (string), like: 1000 carles or 0 root. Logic for swappiness <= 10 Ok, >10 and <= 30 warning, >30 red (alert). Reduced digits for swap to 2, to avoid confusions.
I have updated my book Python 3 Combat Guide, with another full cycle, step by step, to convert an ugly script that escapes to shell to a nice OOP code with Unit Testing, step by step.
I have updated my book ZFS for Ubuntu 20.04 LTS, adding how to create a pool and Datasets for home, sharing NFS for the Media Player.
If you like Star Wars and the Mandalorian, you may laugh has much as I did with this video:
As you see I’m writing more articles about Windows, Mac Os X, and proprietary Software. Some of my colleagues work in companies and use proprietary Software, so I’ll be writing more articles about those ecosystems. I spend more time now with colleagues working on all kind of projects, and with students that have other problems too, so I help them. However my main focus is Open Source, Architecture, Scaling, programming in Python and Java.
Is something very simple, but will help my student friends to validate Input from Keyboard without losing too many hours.
The Input Validation Classes I create in PHP for Privalia or in my PHP Catalonia Framework, are much, much, more powerful, allowing the validation of complete forms, rendering errors, etc… although they were created for Web, and not for Keyboard input.
It recursively goes to all the subdirectories looking for .py files, and then it counts the lines.
I updated the price of my books to be the minimum allowed by LeanPub, to $5 USD, and created a bundle of two of them for $7 USD.
So people can benefit from this during the lock down.
I’ve updated the Python Combat Guide book with a sample of using Paramiko Libraries for SSH, and increased the Object Oriented Programing and Unit Testing, sections. I also added some books to the Bibliography.
I’ve read the postmortem initial analysis from Slack’s incident. It’s really interesting.
I cannot share it, but I guess that at some point they will publish it on their blog:
I’ve published a class to validate input from Keyboard in Java. Is something very simple, thinking basically about university students struggling to Scanner and nextInt() methods, and that can save many hours.
I decided to lower the price of my book to the minimum in LeanPub $5 USD while covid is going on in order to help people with their lives. https://leanpub.com/u/carlesmateo
I read with surprise that Comcast is capping the Internet use to 1.2TB per month, and that they will be charging excess.
So… if I contract a Backup with Carbonite or BackBlaze or DropBox or another company and I backup my 10TB files, Comcast will ruin me charging excesses… Or if I work from home, or the family watches a lot of Netflix… I can only thinK on their Cast Strategy of CastNumberOfClientsToBankrupcy.
A joke to indicate that I think they will loss clients.
Imagine yesterday I downloaded two images of Ubuntu, being 5 GB, installed Call of Duty in one computer 180 GB, installed few Xbox games 400 GB, listened to Spotify 10 Gb, watched youtube 3 GB, watched Netflix 4 GB, so 602 GB in one day.
Not counting the bandwidth WFH (Working from Home).
Not counting Windows Updates, TV updates, consoles updates, Android Updates, Ubuntu updates…
And this is done in the middle of the covid-19 pandemic, with so many people lock down at home, playing video games, watching movies, and requiring desperately distractions.
I think what is most interesting of this article are the explanations about the Memory. Adding a swapfile is something easy, but there is related information that may be of interest for the young Engineers.
Free Games: Epic Games allow you to download these games for free for a limited period of time
However the 12 Gbps SAS SSD were returning Checksum errors in ZFS when I did copy information or I ran scrub. I’m afraid the enclosure can only provide 6 Gbps at max, or a poor connection. Cables or expanders use to be the reason. I ordered new cables to make a direct connection to the HBA Controller without the enclosure to validate my theory and the drives stopped showing errors.
There is something good in all bad: I have been able to document and explain how to troubleshoot, actual errors in ZFS, in my book and talk about the problems with the cables, and the advantages of using a SAS controller even if you use SATA drives.
I got my first Excellent in an Assignment in an Ireland university, which makes me specially happy. And I keep going on studying in Linux Academy, the last course I did was GCP and Terraform, even if I knew both it helps me to keep my skills sharp.
I share with you some offers and charity bundles that I enrolled and enjoyed a lot:
There is an offer with Microsoft Pass which is that we can use Disney+ for free during 30 days.
I started watching the Mandalorian, Season 2, and is wonderfully displayed in 4K. The quality of the video surprised me. Not that many contents in Netflix are 4K and I really enjoyed the great quality of the image.
Humble Bundle offers a pack of 8 VR games per €13.45. If you like Virtual Reality and have your headset, this pack is amazing, and the benefits go to charity: Movember. The games are downloaded from Steam and the pack will last for 14 days.
Don’t forget to balance how much of your contribution goes to every player.
Unfortunately by default most of the money goes to O’Reilly and Humble Tip and few to the Charity cause. You can change that from the web when going to to the payment.
This article talks about how at Riot Games they use Slack. Slack is really a powerful tool, and also makes the communication more human in companies with their approach and the funny icons and /giphy. I’m very serious when it comes to work but I recognize the friendly, warm, human and lovely touch these kind of animated icons bring to the conversations.
Remember that life of the SSD is different from spinning drives. I recommend to keep your backups on external spinning drives disconnected most of the time.
I updated it the Nov-01, as I normally do, bringing more content.
I’ve been paid the royalties for he past two months and I reinvested everything (and more from my pocket) in Hardware for working with ZFS.
I was offered by an editorial in The States to publish Python Combat Guide and other of my books worldwide. I was thinking it for a while. It was very good money, translation to multiple languages and platforms and marketing and a lot of promotion, but I would had loss the rights and the Freedom I have now, like the possibility to offer discount coupons to who I want and to update the contents often. So to celebrate my decision for you, readers of the blog, during September, I provide a discounted price of $5 USD for the fist 100 sales instead of the $25 USD suggested price. Use the following link:
As part of my effort to contributing with nice Open Source products to the Community I have made some investments to keep contributing to:
OpenZFS
My old tool for managing ZFS and Network shares easily
I’m writing a new book about managing ZFS for Small Business too, so I show how to operate on this hardware, good points and downsides.
I’m assembling a new Pc with ZFS plenty of Disk Storage within a mix of:
SAS Enterprise grade SSD 2.5″
SATA 12Gb Enterprise grade SSD 2.5″
SATA SSD 2.5″
SATA HDD 2TB 2.5″
SATA HDD 2TB 3.5″
I’m a big fan of Intel, but this time I have chosen AMD. Concretely a AMD Ryzen 7 3700X AM4 8 Core / 16 Threads, 3.6 GHz to 4.4 GHz with Turbo. The reason I chose this CPU is because it only uses 65W but still has 8 Cores / 16 Threads.
Also I want to see the performance of this AMD Ryzen with CMIPS and another important reason is that AMD motherboards support PCI 4.0. I have bought a NVMe SSD Samsung 980 PRO PCI 4.0 (x4) able to read at 6,400 MB/s. I will use this AMD box for running VMs as well. Basically Virtual Box and Docker.
I’ve been surprised that for 169.99 GBP I can have a very good Asus Motherboard with a 2.5 Gb Ethernet: ASUS ROG STRIX B550-F GAMING, AMD B550, AM4, DDR4, PCIe 4.0, SATA3, Dual M.2, CrossFire, 2.5GbE, USB 3.2 Gen2 A+C, ATX.
In order to have an Asus motherboard with a 2.5 Gb Ethernet for Intel I had to jump to a 254 GBP motherboard and Intel is still PCI 3.0. Actually there are PCI 10Gb NICs at 80 GBP so at some point I’ll upgrade my home network from Gigabit to 10 Gb. That will come slowly, but if the new equipment I assemble has 2.5 Gb when I upgrade the main switches to 10 Gb, at least I’ll be able to communicate at 2.5 Gb without ant additional change.
Also memory at 3200, speed that the AMD motherboard can provide, is more than affordable.
This new server will have 64 GB of RAM (Corsair DDR4 Vengeance PC4-25600 (3200)), as I plan to run VMs and use Volumes mounted via iSCSI and locally as block devices to improve my Software. I’ve bought a new UPS to keep it running in case power goes down. That’s something that doesn’t happen often in my city in Ireland, honestly, but I never forget that this happens in Barcelona two or three times per year, and that a high tension spike can burn your motherboard, drives, or electronics like the TV or the fridge. I’ve bought as well a new KVM Switch, a HDMI 4K and USB too one, so I don’t have to have so many keyboards. My logitech M720 allowed me to use it with 3 computers, but still I want something more operational. The KVM I bought allow me to switch with a button or within a hotkey in the keyboard.
I bought a new Icy box fox handling 6 2.5 drives in just one bay of the tower, and a 850 Watt Corsair PSU that will be able to power the many drives I want at the same time.
ZFS on Ubuntu 20.04.1 LTS A guide for Small/Medium Business and power users to work with ZFS. https://leanpub.com/zfs-ubuntu
Those can be purchased while I’m still working on them and get the updates that I’ll be publishing and keeping a communication with me about doubts or improvements.
Halloween Software Offers
I saw some Halloween offers and I purchased Software licenses for Software I use.
I contribute a lot to Open Source, and many years ago before Open Source existed I was creating Freeware Software. But I think that good commercial Software deserves to be supported. Like everything in life, if they are doing a good work that is useful to me, why not giving them support?. It is also a way to make sure they will continue producing amazing Software. And in the other hand, myself, I create Software. Some times commercial Software, and I like to be paid, so I apply the same principle.
One of my colleagues showed me dstat, a very nice tool for system monitoring, and bandwidth of a drive monitoring. Also ifstat, as complement to iftop is very cool for Network too. This functionality is also available in CTOP.py
As I shared in the past news of the blog, I’m resuming my contributions to ZFS Community.
Long time ago I created some ZFS tools that I want to share soon as Open Source.
I equipped myself with the proper Hardware to test on SAS and SATA:
12G Internal PCI-E SAS/SATA HBA RAID Controller Card, Broadcom’s SAS 3008, compatible for SAS 9300-8I. This is just an HDA (Host Data Adapter), it doesn’t support RAID. Only connects up to 8 drives or 1024 through expander, to my computer. It has a bandwidth of 9,600 MB/s which guarantees me that I’ll be able to add 12 SAS SSD Enterprise grade at almost the max speed of the drives. Those drives perform at 900 MB/s so if I’m using all of them at the same time, like if I have a pool of 8 + 3 and I rebuild a broken drive or I just push Data, I would be using 12×900 = 10,800 MB/s. Close. Fair enough.
VANDESAIL Mini-SAS Cables, 1m Internal Mini-SAS to 4x SAS SATA Forward Breakout Cable Hard Drive Data Transfer Cable (SAS Cable).
SilverStone SST-FS212B – Aluminium Trayless Hot Swap Mobile Rack Backplane / Internal Hard Drive Enclosure for 12x 2.5 Inch SAS/SATA HDD or SSD, fit in any 3x 5.25 Inch Drive Bay, with Fan and Lock, black
Terminator is here. I ordered this T-800 head a while ago and finally arrived.
Finally I will have my empty USB keys located and protected. ;)
I have benchmarked three different CPUs and two Compute optimized Amazon AWS instances with CMIPS 1.0.5 64bit. The two Intel Xeon baremetals equip 2 x Intel Xeon Processor and the third baremetal equips a single Intel Core i7-7800X:
If you’re surprised by the number of cores reported by the Amazon instance m5d.24xlarge, and even more for the baremetal c5n.metal, you’re guessing well that this comes from having Servers with 4 CPUs for Compute Optimized series.
When I can choose I use Linux, but in many companies I work with Windows workstations. I’ve published a list of useful Software I use in all my Windows workstations.
WFH I currently use two external monitors attached to the laptop. I planned to add a new one using a Display Port connected to the Dell USB-C dongle that provides me Ethernet and one additional HDMI as well. I got the cable from Amazon but unfortunately something is not working. In order to make myself comfortable and see some the graphs of the systems worldwide as I have on the office’s displays, I created a small HTML page, that joins several monitor pages in one single web page using frames. This way I only have one page loaded on the browser, maximized, and this monitor is dedicated to those graphs of the stats of the Systems. Something very simple, but very useful. You can extend the number of columns and rows it to have more graphics in the same screen.
If you don’t have the space or the resources for more monitors you can use the ingenious.
I have a cheap HDMI switch that allows me to do PinP (Picture in Picture) with one main source on the monitor, and two using a fraction of their original space. It may allow you to see variants in graphics.
And in you have only a single monitor, you can use a chrome extension that rotates tabs, which is also very useful.
Be careful if you use the reload features with software like Jira or Confluence. If they are slow normally, imagine if you mess it by reloading every 30 seconds… I discourage you to use auto refresh on these kind of Softwares.
This past week I have connected the XBOX One X Controller to the Windows laptop for the first time. Normally I use the Pc only for strategy games, but I wanted to play other games like Lost Planet 3, or Fall Guys in a console mode way. I figured that would be very easy and it was. You turn on the controller, press the connect button like you did to pair with the console, and in Windows indicate pair to a Xbox One controller. That’s it.
I’ve also updated my Python 3 Combat Guide, to add the explanation, step by step, about how to refactor and make resilient, and add Unit Testing to a spaghetti code, and turn it into a modern OOP. Is currently 255 DIN-A4 pages.
This is something I wanted to share with you for a while. One of the most funny things in my career is what I call: Squirrel Strikes Back
I named this as the first incident where a provider told that the reason of a fiber failure was a squirrel chewing the cable.
I popularized this with my friends in Systems Administration and SRE and when they suffer a Squirrel Attack incident, they forward it to me, for great joy.
I’m used to construction or gas, water, electricity, highways repair operations on the cities accidentally cutting fiber cables, thunders or truck accidents on the highway breaking the floor and cutting tubes and issues like that. I’ve been seeing that for around 25 years.
So the first time I saw a provider referring to a squirrel cutting the cables it was pretty hilarious. :)
In my funny mental picture: I could visually imagine a cable thrown in the middle of the forest, over trees, and a squirrel chewing it as it tastes like peanuts. :) or a shark cutting a Google’s or Facebook’s intercontinental cable thrown without any protection. ;)
The sense of humor and the good vibes, are two of the most important things in life.
