Tag Archives: VirtualBox

News from the blog 2020-11-03

Nice articles recommended

This article talks about how at Riot Games they use Slack. Slack is really a powerful tool, and also makes the communication more human in companies with their approach and the funny icons and /giphy. I’m very serious when it comes to work but I recognize the friendly, warm, human and lovely touch these kind of animated icons bring to the conversations.

Remember that life of the SSD is different from spinning drives. I recommend to keep your backups on external spinning drives disconnected most of the time.

Operating at Scale – An Inside Look at Facebook’s Production Engineering Team

CMIPS

I’ve been working on testing performance of more configurations on Azure and GCP.

I’m also looking forward to test the AMD Ryzen™ 7 3700X, AM4, Zen 2, 8 Core, 16 Thread, 3.6GHz, 4.4GHz Turbo that is arriving to me this week.

CTOP.py v. 0.7.8 released

I closed the ticket #21 (Thank Jian!) so ensuring CTOP.py is compatible with Python 3.5 versions.

Feature requests and bugs are listed using gitlab: https://gitlab.com/carles.mateo/ctop/-/issues

My Python Combat Guide Book

I updated it the Nov-01, as I normally do, bringing more content.

I’ve been paid the royalties for he past two months and I reinvested everything (and more from my pocket) in Hardware for working with ZFS.

I was offered by an editorial in The States to publish Python Combat Guide and other of my books worldwide. I was thinking it for a while. It was very good money, translation to multiple languages and platforms and marketing and a lot of promotion, but I would had loss the rights and the Freedom I have now, like the possibility to offer discount coupons to who I want and to update the contents often. So to celebrate my decision for you, readers of the blog, during September, I provide a discounted price of $5 USD for the fist 100 sales instead of the $25 USD suggested price. Use the following link:

https://leanpub.com/pythoncombatguide/c/blog-carles-nov2020

ZFS progress

As part of my effort to contributing with nice Open Source products to the Community I have made some investments to keep contributing to:

  • OpenZFS
  • My old tool for managing ZFS and Network shares easily

I’m writing a new book about managing ZFS for Small Business too, so I show how to operate on this hardware, good points and downsides.

I’m assembling a new Pc with ZFS plenty of Disk Storage within a mix of:

  • SAS Enterprise grade SSD 2.5″
  • SATA 12Gb Enterprise grade SSD 2.5″
  • SATA SSD 2.5″
  • SATA HDD 2TB 2.5″
  • SATA HDD 2TB 3.5″

I’m a big fan of Intel, but this time I have chosen AMD. Concretely a AMD Ryzen 7 3700X AM4 8 Core / 16 Threads, 3.6 GHz to 4.4 GHz with Turbo. The reason I chose this CPU is because it only uses 65W but still has 8 Cores / 16 Threads.

Also I want to see the performance of this AMD Ryzen with CMIPS and another important reason is that AMD motherboards support PCI 4.0. I have bought a NVMe SSD Samsung 980 PRO PCI 4.0 (x4) able to read at 6,400 MB/s. I will use this AMD box for running VMs as well. Basically Virtual Box and Docker.

I’ve been surprised that for 169.99 GBP I can have a very good Asus Motherboard with a 2.5 Gb Ethernet: ASUS ROG STRIX B550-F GAMING, AMD B550, AM4, DDR4, PCIe 4.0, SATA3, Dual M.2, CrossFire, 2.5GbE, USB 3.2 Gen2 A+C, ATX.

In order to have an Asus motherboard with a 2.5 Gb Ethernet for Intel I had to jump to a 254 GBP motherboard and Intel is still PCI 3.0. Actually there are PCI 10Gb NICs at 80 GBP so at some point I’ll upgrade my home network from Gigabit to 10 Gb. That will come slowly, but if the new equipment I assemble has 2.5 Gb when I upgrade the main switches to 10 Gb, at least I’ll be able to communicate at 2.5 Gb without ant additional change.

Also memory at 3200, speed that the AMD motherboard can provide, is more than affordable.

This new server will have 64 GB of RAM (Corsair DDR4 Vengeance PC4-25600 (3200)), as I plan to run VMs and use Volumes mounted via iSCSI and locally as block devices to improve my Software. I’ve bought a new UPS to keep it running in case power goes down. That’s something that doesn’t happen often in my city in Ireland, honestly, but I never forget that this happens in Barcelona two or three times per year, and that a high tension spike can burn your motherboard, drives, or electronics like the TV or the fridge. I’ve bought as well a new KVM Switch, a HDMI 4K and USB too one, so I don’t have to have so many keyboards. My logitech M720 allowed me to use it with 3 computers, but still I want something more operational. The KVM I bought allow me to switch with a button or within a hotkey in the keyboard.

I bought a new Icy box fox handling 6 2.5 drives in just one bay of the tower, and a 850 Watt Corsair PSU that will be able to power the many drives I want at the same time.

More books coming

I started two new books:

Those can be purchased while I’m still working on them and get the updates that I’ll be publishing and keeping a communication with me about doubts or improvements.

Halloween Software Offers

I saw some Halloween offers and I purchased Software licenses for Software I use.

Backup Guard is one of the products I registered:

https://backup-guard.com/

I contribute a lot to Open Source, and many years ago before Open Source existed I was creating Freeware Software. But I think that good commercial Software deserves to be supported. Like everything in life, if they are doing a good work that is useful to me, why not giving them support?. It is also a way to make sure they will continue producing amazing Software. And in the other hand, myself, I create Software. Some times commercial Software, and I like to be paid, so I apply the same principle.

Programs I use for Windows in my Workstations

I love Linux and Linux tools and I’m a big fan of it, using it for Servers since 1995.

However some companies use Windows for the Workstations, and that’s not necessarily bad.

So I describe here the tools I use to maximize productivity.

Antivirus

That really depends on my employers. I’ve my opinion about several of them.

Apache Directory Studio

For working with LDAP.

BalenaEtcher

To flash images to USB and external drives. https://www.balena.io/etcher/

I also use Rufus https://rufus.ie/

Chrome

Debut
This is a Commercial Software to capture video. I record bugs, tutorials, internal web training sessions…

Docker Desktop for Windows

But not in the laptops cause the hyper-v may conflict with the BitLocker drive encryption and may cause the entire drive to be lost.

However as much as possible, I will do everything in Linux Workstations and Servers.

(CRLF problems in Docker Linux are horrible)

Editplus

Very powerful for doing replacement over large CSV files.

Firefox

Filezilla

GIMP

HeidiSQL

Free Database Manager for MariaDB, MySQL, SQLite, PostgreSQL and Microsoft SQL.

Is compatible with Wine, so you can use it on Linux.

LastPass (for Teams)

With Chrome’s plugin.

LibreOffice

MobaXTerm

With zmodem, sftp, SSH, tunnel….

MyDebugger

A MySQL debugger. Specially useful for Stored Procedures.

Is compatible with Wine, so you can use it on Linux.

OBS

https://obsproject.com/ is a screen recorder Software, for Linux, Mac Os X and Windows.

One Note

OpenVPN

Opera

Specially useful the option of using a VPN.

Project (Microsoft Project)

PuTTy

PyCharm, PHPStorm, CLion from JetBrains

Python

Radmin

A powerful Server Remote Control for Windows, much more stable than VNC.

Remote Desktop Connection

Rufushttps://rufus.ie

To toast ISOs to USB.

Slack

Toad for Oracle

Videopad

Video editor

VirtualBox
Yes, I always have a Linux VM.

Visio

For the Diagrams.

VLC

Video player

VMWare Player

VNC

Normally RealVNC.

WinRAR

Zoom

News from the blog 2020-08-19

  • I assisted to the OpenZFS leadership meeting.

I tried to continue following it since I left Sanmina. ZFS is really an amazing Software and it’s lead by an amazing Community of super cool Engineers and companies. I would like to continue contributing ASAP.

I bought some new hard drives in order to work a bit on this. You don’t need to have dedicated hardware if you want to test features. You can run in a VirtualBox or VMWare Workstation.

  • I received more books about DevOps and Python

None is perfect. I see flaws in all of them and bad architecture practices*, however from all I learn interesting things.

*I guess that’s why I wrote my own book :)

You know, I study every day. At least 30 minutes, after work. As part of my healthy routines.

But I also study and learn during the work, as we have time available for this.

I’m very fortunate that Blizzard gives me time every day to study. That’s amazing. They also send us to events paying the ticket, travel, hotel, expenses… now with covid-19 we only go to virtual events, but the company still pay for this and give free days. Is a very nice company.

I use a lot Linux Academy too:

I continue having purchases of my book, and I’m very happy about that. I’m working on improving it and providing more contents and samples going from the scratch, with step by step code samples. From spaghetti code reading CSV files, to OOP with Full Coverage.

  • My application for a Higher degree Computer Science Cloud Computing (Level 8) has been accepted. The Irish government pays me 90% of the degree, and Blizzard will pay me the other 10% after I pass the first year course.

I’m really grateful to this beautiful country, Ireland.

Having an Irish degree is something that brings me an special illusion.

  • I have updated CTOP.py with some interesting features

It allows to pass a fixed width and height for the terminal render. That’s very useful when you run CTOP in a Docker non interactive session, or from a Cron, with the –iterations=1 so the output can be captured programmatically.

  • Jetbrains has provided me with a Free License of all their products, in order to support my work in Open Source projects. That’s very nice. I’m using now mainly PyCharm and PhpStorm.
  • At the beginning of the covid-19 I wrote a simulator in Python. That’s why I was able to anticipate that the number of cases and deaths would be very much higher when nobody around me knew what was going to happen. My first simulations were simple, and the algorithms were growing in complexity until I had a full rich Object Oriented modeler. Maybe I’ll write an article about this someday.
    • I based my data in https://www.worldometers.info/coronavirus/#countries
    • I studied the evolution of several countries and I was working with simulations in Spain until their government started blocking the information and stop providing transparent and accurate metrics.
  • I’m seeing how the covid is affecting and transforming several kind of business:
    • Meetup.com I see meetups with more than 1,000 users closing, as they are no meeting anymore
    • Airlines, obviously
    • Hotels, offering less services
    • Metasearchers and OTAs (Online travel agencies)
    • I can imagine the impact on airbnb
    • Discos, nightclubs are closing doors
    • Restaurants, they will lose the Christmas season (with families and companies doing lunch and dinners)
  • At the same time, other companies are hitting records in sales
  • After doing a Masterclass to some colleagues about Refactor, Code Reliability, Quality, The non-happy path and Unit Testing, I’m preparing some contents that I’ll publish to the Community soon. So far I created this repo, where I added the source code for lesson 0: starting to program in Python videos that I created few months ago to help beginners.

https://gitlab.com/carles.mateo/teach-unit-testing/-/tree/master/lesson0

I also added some contents to lesson 1, where we refactor pure spaghetti code with no error control, to something more elaborated with unit testing and full code coverage. Still procedural, but I will jump to next class in two weeks, where we will move to OOP and Dependency Injection.

  • Here my “Harley” assembled :)

Creating a VM for compiling ZFS with RHEL6.10

As you know I created the DRAID project, based in ZFS.

One of our customers wanted a special custom version for their RHEL6.10 installation with a custom Kernel.

This post describes how to compile and install ZFS 7.x for RHEL6.

First create a VM with RHEL6.10. Myself I used Virtual Box on Ubuntu.

If you need to install a Custom Kernel matching the destination Servers, do it.

Download the source code from ZFS for Linux.

install the following packages which are required by zfs compiler:

sudo yum groupinstall "Development Tools"
sudo yum install autoconf automake libtool wget libtirpc-devel rpm-build
sudo yum install zlib-devel libuuid-devel libattr-devel libblkid-devel libselinux-devel libudev-devel
sudo yum install parted lsscsi ksh openssl-devel elfutils-libelf-develsudo yum install kernel-devel-$(uname -r)

Steps to compile the code:

1- Make sure  the zfs file exists under zfs/contrib/initramfs/scripts/local-top/

if not exists, create a file called zfs  under zfs/contrib/initramfs/scripts/local-top/  and add the following to that file:

#!/bin/sh
PREREQ=”mdadm mdrun multipath”

prereqs()
{
       echo “$PREREQ”
}

case $1 in
# get pre-requisites
prereqs)
       prereqs
       exit 0
       ;;
esac


#
# Helper functions
#
message()
{
       if [ -x /bin/plymouth ] && plymouth –ping; then
               plymouth message –text=”$@”
       else
               echo “$@” >&2
       fi
       return 0
}

udev_settle()
{
       # Wait for udev to be ready, see https://launchpad.net/bugs/85640
       if [ -x /sbin/udevadm ]; then
               /sbin/udevadm settle –timeout=30
       elif [ -x /sbin/udevsettle ]; then
               /sbin/udevsettle –timeout=30
       fi
       return 0
}


activate_vg()
{
       # Sanity checks
       if [ ! -x /sbin/lvm ]; then
               [ “$quiet” != “y” ] && message “lvm is not available”
               return 1
       fi

       # Detect and activate available volume groups
       /sbin/lvm vgscan
       /sbin/lvm vgchange -a y –sysinit
       return $?
}

udev_settle
activate_vg

exit 0

make the created zfs file executable:

chmod +x  zfs/contrib/initramfs/scripts/local-top/zfs

2-  inside  draid-zfs-2019-05-09 folder, execute the following commands:execute Auto generate script:

./autogen.sh

execute configuration script:

./configure

Please note we use this specific configuration for bettter results:

./configure –disable-pyzfs –with-spec=redhat

create rpms:

make rpm

remove all test rpms:

rm zfs-test*.rpm

3- install all created rpms

yum install *x86_64* -y

4- verify that zfs is been installed

zfs

this command will display zfs help. 

Another interesting trick I instructed my Team to do is to add a version number to zfs, with a parameter -v or –version.

So if you want to do the same, you have to edit:

zfs/cmd/zfs/zfs_main.c

Under:

cmdname = argv[1];

In my code is line 7926, then add:

/* DRAIDTEAM - added new command to display zfs version*/
if ((strcmp(cmdname, "-v") == 0) || (strcmp(cmdname, "--version") == 0)) {
    (void) fprintf(stdout, "0.7.0_DRAID-1.2.9.08021755\n");
    return (0);
}

You can check the Kernel Module info by using modinfo zfs, but I found it handy to allow to just do:

zfs -v