This is a great new for scaling performance in the Data Centers. For routers, switches…
And this makes me think about all the Architects that are using Memcached and Redis in different Servers, in Networks of 1Gbps and makes me want to share with you what a nonsense, is often, that.
So the idea of having Memcache or Redis is just to cache the queries and unload the Database from those queries.
But 1Gbps is equivalent to 125MB (Megabytes) per second.
Local RAM Memory in Servers can perform at 24GB and more (24,000,000 Megabytes) per second, even more.
A PCIE NVMe drive at 3.5GB per second.
A local SSD drive without RAID 550 MB/s.
A SSD in the Cloud, varies a lot on the provider, number of drives, etc… but I’ve seen between 200 MB/s and 2.5GB/s aggregated in RAID.
In fact I have worked with Servers equipped with several IO Controllers, that were delivering 24GB/s of throughput writing or reading to HDD spinning drives.
If you’re in the Cloud. Instead of having 2 Load Balancers, 100 Front Web servers, with a cluster of 5 Redis with huge amount of RAM, and 1 MySQL Master and 1 Slave, all communicating at 1Gbps, probably you’ll get a better performance having the 2 LBs, and 11 Front Web with some more memory and having the Redis instance in the same machine and saving the money of that many small Front and from the 5 huge dedicated Redis.
The same applies if you’re using Docker or K8s.
Even if you just cache the queries to drive, speed will be better than sending everything through 1 Gbps.
This will matter for you if your site is really under heavy load. Most of the sites just query the MySQL Server using 1 Gbps lines, or 2 Gbps in bonding, and that’s enough.
You do df -h or ls / and the terminal freezes and not even CTRL + C works, you have a lock.
Normally this is due to a lock of the system trying to perform an IO.
Could be a physical spinning disk failing, but the most probably nowadays is that you have a network mount point and it is timing out.
If you execute mount and you get a timeout, and when you finally see the list you see a NFS, iSCSI or another kind of Network mount (you will see an Ip Address), check for errors.
To do this in CentOS/RHEL you can do as root:
dmesg | grep -i "timed"
or depending on the System
cat /var/log/messages | grep -i "timed"
You’ll get something like this:
[root@compute01 carles]# dmesg -T | grep timed | head -n5
[Fri Mar 20 02:27:44 2020] nfs: server storage07 not responding, timed out
[Fri Mar 20 02:27:44 2020] nfs: server storage07 not responding, timed out
[Fri Mar 20 02:27:44 2020] nfs: server storage07 not responding, timed out
[Fri Mar 20 02:27:44 2020] nfs: server storage07 not responding, timed out
[Fri Mar 20 02:27:45 2020] nfs: server storage07 not responding, timed out
Please note I use dmesg -T in order to have human readable date instead of Unix Epoch.
Please note: Even if I tried to make it easy, probably there are too many concepts for a non-programmer. Will try to deliver more basic previous knowledge and foundations, so people with zero knowledge don’t feel overwhelmed.
Start by installing Python 3.8 or 3.9 in your computer, and the IDE PyCharm. Install also Git, and create an account in GitLab so you can share code with other people and understand how Git works.
Ok, so you can take a look at my video, and hopefully it makes spark your motivation to learn by yourself. :)
I’ve been asked why I used print(“”) instead of print().
Is a good question. The reason is, when we programmed in Python 2.x the native way was to print without parenthesis, like:
print "Hello World!"
Python 3.x was incompatible with that and requires to use parenthesis, like:
print("Hello World!")
Fortunately Python 2.x accepts also to print using parenthesis. In order to have compatibility within Python 2.x and Python 3.x or for future compatibility we were using always print(“Whatever”) in Python2.
However, there is one difference.
If you user print() or print(“”) in Python3 that will generate an empty line.
In Python 2 print(“”) will generate too an empty line, nevertheless print() in Python2 will print two parenthesis. We don’t want that.
This is illustrated in this screenshot:
So all the people that are at home, closed down for coronavirus, you have a chance now to start learning Python and from there get a live as programmer.
You can download the code for this lesson 0, from:
In order to be able to do more samples, and then being a bit interesting an dynamic, I will introduce here how to get data inputted by the Keyboard.
print("Please enter your name:")
s_name = input()
This will add whatever we type, without the final Enter, to the String variable s_name.
Capturing numbers from Keyboard
How we do to capture a number, like how old are you, in years?.
The same way, and then we convert this to an Integer value. An Integer is a data type which is basically a number, not decimal. Like: 1, 2, 7, 1000 o -5.
print("Please enter your name:")
s_name = input()
print("Please enter your age:")
s_age = input()
# With int() we convert a String to an Integer, as long as it is possible.
# Wit str() we convert a Integer to a String, as long as it is possible.
i_age = int(s_age)
If you enter a number incorrectly and so that cannot be converted, you will get an Exception Error. That is something that happened in a way that was not expected. These error can be trapped, and we will see this later, in the future.
You know:
How to capture data from the keyboard with input()
How to convert data entered as String to Integer with int()
How to sum two numbers, like 2 + 3
How to subtract two numbers, like 2 – 3
How to multiply, like 2 * 3
So know, you should be able to solve a basic arithmetic exercise in Hacker Rank:
I’m teaching Unit Testing, Refactors, Quality Code and moving from Procedural to OOP to some colleagues, you can find source code for our classes here (please, be aware that there are some error made on purpose to show why and why not do things and hot to apply proper unit testing)
I was using a lot Amazon. Sending parcels to my previous job offices, and now to Blizzard offices, so I subscribed to Amazon Prime. With COVID-19 virus we were sent to do Remote Work, and now with the lock down basically I’m 99.99% of the time at home.
I did a test to see how it works sending to home during the pandemic.
I choose two different items, I reviews the order, they were going to be delivered separately, one day of distance.
I choose two items that will fit in my mailbox, separated or together. One USB3 3mts male female and a Blu-ray movie.
My surprise comes when I go to the mailbox one day before and I see that I have a paper from an-post telling that they pass by to deliver my parcel, and they did not leave because it doesn’t fit the mailbox and they did not want to leave it a common space. For my surprise both Amazon parcels were grouped and sent before time. Maybe in a bigger box. But the mailman did not ring my door.
The paper tells me to get my parcel in the middle of the city, during the lock down. No way! I’m not going to risk my health and specially from elders, just to grab a cable and a movie.
I had the chance to request re-delivery to an Post, so I do. I fill all the info, I inform my phone number, email, I indicate which door to ring, and two days after as promised… a paper from an Post!.
They did not even rang my bell again.
I go to Amazon to cancel the order, but the process is only created for if you got the items.
Fuck it. I’m not going to order anything else to Amazon until that COVID-19 passes.
I don’t know if the postman just avoids people for fear to contagion or the process of an Post is awful and he didn’t get any information. But I’ll not buy anything even if I cannot buy in other places cause the lock down.
I was going to maintain my Amazon Prime subscription, even if I know that I’ll not use it much with the lock down, but makes no sense. Also:
I use Netflix and my Raspberry Pi 4, I was not using Amazon Prime Video.
I use Spotify, I was not using Amazon Prime Music.
I like to read in paper, not in eBook, so I was not using the eReader options.
I’ve been working for years within Data centers, with D&R strategies, and then in the middle of COVID-19, with huge demands on increments of bandwidth and compute, some DCs decided to do not allow in the Engineers of their customers.
As somebody that had my own Startup and CSP and had infrastructure in DCs and servers from customers in colocation, and has replaced Hw components at 1AM, replaced drives from broken RAIDs, and fixed systems so many times inside so many Datacenters across the world, I’m shocked about that.
I understand health reasons can be argued, but I still have Servers in Datacenters because we all believed they were the most safe place, prepared for disaster and recovery, with security, 24×7… and now, one realise that cannot enter to fix or upgrade the own machines.
Please note, still you can use the remote hands from the DC, although this is not a good idea many times, I’m not sure this will still be an available option when the lock down in those countries becomes more strict.
I’m wondering if DCs current model have any future at all.
I think most of the D&R strategies from now will be in the cloud, in different regions, with different providers, so companies can resist providers or governments letting them down.
Quick Access to my selection Last Update: 2022-10-24 09:36:42 Ireland Time / 2022-10-24 01:36:42 PDT-0700 Unix epoch: 1666600602
CQLSÍ a wrapper to use Cassandra from PHP
Written in 2014, a time when there were no drivers for PHP. Published: 2014 Views: 6,512 views
Cassandra Universal Driver
A HTTP gateway for all the languages supporting curl/sockets. Written in 2014, a time when there were no drivers for many languages. Published: 2014 Views: 6,573 views
count_repeated_pattern_in_logs.sh
A easy way to see errors that are repeating, e.g.: NFS/iSCSI timeouts. Published: 2020-May Views: 10,529 views
backup_partition_in_files.sh
Compressing an unmounted partition to a image file while compressing on the fly, and breaking into 1GB gz files.
Also explains in a funny way about STDIN, STDOUT, STDERR and methodology investigating in deep. Published: 2020-May Views: 11,147 views
iostat_bandwidth.sh
See the aggregated bandwidth used by all the drives, and the maximum speed achieved. Published: 2020-Aug Views: 9,689 views
count_lines_of_code.sh
Count the lines in .py Python source files recursively and displays individual and total results. Published: 2021-Jan Views: 9,390 views
One line script to log the temperature of HDDs and CPUs in Ubuntu Published: 2021-Jan Views: 8,572 views
Just installed a media player in my Raspberry Pi 4
So I mentioned it was one of my pending tasks, to do while I’m confined here, at home, to help the Irish government to stop the quick spread of the coronavirus.
I’m happy that the situation in Ireland has stabilized, unlikely in Spain, where that historical lack of discipline and selfishness and super ego to believe Madrid the capital of the world, and so deciding not to close it for quarantine, will cause a lot of pain. I hope the closing of frontiers in Catalonia works.
Well, what I do you’re probably asking yourself, so I installed LibreELEC https://libreelec.tv/.
They have a very nice SD image writer for Linux, Mac and Windows, that will install the proper image on the micro-SD for your ARM device.
This Raspberry Pi 4 comes with Wifi integrated and a Gigabit Ethernet network port.
When I was in Barcelona, I had Kodi with Raspberry pi 2 and version 3.
This model v. 4 is much more cooler. I bought the 4GB version, and has 2xHDMI 4K.
So it is great to connect to any modern TV.
In Barcelona, I have Linux tower as NFS Server sharing my files with the Pi. Work good, even for the 100Mbit NIC of the version 3, but at that time I was only playing Full HD as the Pi didn’t supported greater resolution, and I only had that resolution on my displays too.
For now, I’m going to explore how is reading from a USB 3.0. Let’s see if it’s able to play smoothly.
The cool thing also is that I have SSH access, and so I can use the Pi for many more things. :)
I have my first update, I noticed that copying to that USB was not the best for me, as I tried to copy a .MKV file of 4.9GB and I encountered the limit of 4GB of FAT32. I could format the USB as ext4, but what I did is, SSH into the box, I see that I have two partitions on the SD for booting the Pi, the second one is a ext4 called storage. So I copied to the SD, through the network, using sftp the file I wanted.
The Gigabit connection was fast, but when the buffer fulled it started to show the real speed of the SD which is 15MB/s for writing.
Ext4 has no problem in holding a file 4.9GB so I’m watching my movie now. Will think about setting a NFS for the Pi as it will be very convenient. :)
I have an external, remote, keyboard logitech, but it happens that LibreELEC recognizes my Sony command, from the television. I don’t need the keyboard/mouse. Nice.
Here you can see my Raspberry Pi 4, connected to TV, in “combat mode”, naked, as PoC, before setting in its definitive place behind the TV.
Playing from the external USB 3.0 stick was also fluid, allowing 4K perfectly.
The only problem I has was when I was pushing movies to the USB through the network, and playing at the same time from the SD. It seems like the Raspberry reached its limits doing this and playing stuck frequently.
After years in which many Engineers requested to the companies to be able to Remote Work, with most of answers No, now it happens that not only is good for the company, is the only way to ensure continuity of business, of many businesses.
One of my colleagues from Denmark, which government has shutdown the country by sending all the public servants to home, in order to prevent the spread of the coronavirus, told me:
“Yes, remote working is here, but has been necessary the four horsemen of the apocalypse”
It is curious, how Remote Working has arrived, no thanks to that was obvious, but due to external emergencies. And I’m glad that my company was prepared for business continuity.
I’ll be staying home, working remotely, in order to contribute to non-spreading the virus, specially among old people. I’m perfectly healthy but that’s a use case, many people will not develop the symptoms and still be able to spread to others.
So I have some plans related to technology to do at home, including few improvements to the blog. What are your plans?.
Update: 2020-03-13 23:16 UTC I’m thinking in all those business which are forces to close, and all the employees that will not get a salary, or will be fired, or will get a salary and the business owner maybe ends in bankrupt as is paying the salaries and no income is being generated.
Update: 2020-03-19 10:58 UTC Some of my friends, even in Human Resources/Recruiting, are starting to remote work for first time. So here is some advice:
I would recommend to get an external monitor, at least 22″, so you neck is not forcing position looking low and your eyes don’t suffer, good light (don’t in dark), a nespresso can be a good friend in the morning, and to have your hands and arms aligned correctly so you don’t suffer from a bad position. Watch the position of the wrists, your arms should be comfortably at the same level than the table, similar in an L, and your eyes be aligned to the top of your monitor. Finally I would recommend to follow a routine, like if you were going to work, so dress like you would do. Don’t stay at home all day in pijamas! ;)
2020-03-06 Heya, I’m doing a set of improvements to the blog.
One, you can already see. I added a new section to the CSS @media, so now screens bigger than 1,800 px in width, will use that width for rendering the page. The original WordPress theme at 960x was too small for our current screens. I will add a new CSS @media for 4K screens promptly.
Other is about the organization of the content. I want to separate a bit the contents, now articles are sequential and is difficult to discover nice contents if they have 2 or more articles more recent, so I will group articles by content and provide a small index on the top page. Also I will provide more areas for Operations, SRE, where it will be easy to locate code, scripts, tricks… things that are useful to our day to day. I also want to make visible the articles about living in different cities, for IT Engineers, with useful tricks and tips. And keep the more complex and more interesting Engineering matters in the main page.
Updates
2020-03-13 15:49 Added SSL to the blog
With more delay I wanted, I bought a SSL certificate, configured Apache, and after few changes to the blog has been set. One very annoying is that WordPress linked the images statically pointing to http://blog.carlesmateo.com so I changed the latest article’s images to point to relative path so they will work nice with http or https.
My reflection is that everything negative can have its positive output. With this coronavirus thing, I decided to focus into improving things. And so I’m doing. :)
ctop.py is an Open Source tool for Linux System Administration that I’ve written in Python3. It uses only the System (/proc), and not third party libraries, in order to get all the information required. I use only this modules, so it’s ideal to run in all the farm of Servers and Dockers:
os
sys
time
shutil (for getting the Terminal width and height)
The purpose of this tool is to help to troubleshot and to identify problems with a single view to a single tool that has all the typical indicators.
It provides in a single view information that is typically provided by many programs:
top, htop for the CPU usage, process list, memory usage
meminfo
cpuinfo
hostname
uptime
df to see the free space in / and the free inodes
iftop to see real-time bandwidth usage
ip addr list to see the main Ip for the interfaces
netstat or lsof to see the list of listening TCP Ports
uname -a to see the Kernel version
Other cool things it does is:
Identifying if you’re inside an Amazon VM, Google GCP, OpenStack VMs, Virtual Box VMs, Docker Containers or lxc.
Compatible with Raspberry Pi (tested on 3 and 4, on Raspbian and Ubuntu 20.04LTS)
Uses colors, and marks in yellow the warnings and in red the errors, problems like few disk space reaming or high CPU usage according to the available cores and CPUs.
Redraws the screen and adjust to the size of the Terminal, bigger terminal displays more information
It doesn’t use external libraries, and does not escape to shell. It reads everything from /proc /sys or /etc files.
Identifies the Linux distribution
Supports Plugins loaded on demand.
Shows the most repeated binaries, so you can identify DDoS attacks (like having 5,000 apache instances where you have normally 500 or many instances of Python)
Indicates if an interface has the cable connected or disconnected
Shows the Speed of the Network Connection (useful for Mellanox cards than can operate and 200Gbit/sec, 100, 50, 40, 25, 10…)
It displays the local time and the Linux Epoch Time, which is universal (very useful for logs and to detect when there was an issue, for example if your system restarted, your SSH Session would keep latest Epoch captured)
No root required
Displays recent errors like NFS Timed outs or Memory Read Errors.
You can enforce the output to be in a determined number of columns and rows, for data scrapping.
You can specify the number of loops (1 for scrapping, by default is infinite)
You can specify the time between screen refreshes, for long placed SSH sessions
You can specify to see the output in b/w or in color (default)
Plugins allow you to extend the functionality effortlessly, without having to learn all the code. I provide a Plugin sample for starting lights on a Raspberry Pi, depending on the CPU Load, and playing a message “The system is healthy” or “Warning. The CPU is at 80%”.
Limitations:
It only works for Linux, not for Mac or for Windows. Although the idea is to help with Server’s Linux Administration and Troubleshot, and Mac and Windows do not have /proc
The list of process of the System is read every 30 seconds, to avoid adding much overhead on the System, other info every second
It does not run in Python 2.x, requires Python 3 (tested on 3.5, 3.6, 3.7, 3.8, 3.9)
I decided to code name the version 0.7 as “Catalan Republic” to support the dreams and hopes and democratic requests of the Catalan people, to become and independent republic.
I created this tool as Open Source and if you want to help I need people to test under different versions of:
Atypical Linux distributions
If you are a Cloud Provider and want me to implement the detection of your VMs, so the tool knows that is a instance of the Amazon, Google, Azure, Cloudsigma, Digital Ocean… contact me through my LinkedIn.
Monitoring an Amazon Instance, take a look at the amount of traffic sent and received
Some of the features I’m working on are parsing the logs checking for errors, kernel panics, processed killed due to lack of memory, iscsi disconnects, nfs errors, checking the logs of mysql and Oracle databases to locate errors
