But he had problems installing napalm-base package.
Note that the package is no longer maintained.
He tried with the last one, and with the previous one (0.25.0), but he always got the error: ModuleNotFoundError: No module named ‘pip.req’
pip3 install napalm-base==0.25.0
Defaulting to user installation because normal site-packages is not writeable
Using cached napalm-base-0.25.0.tar.gz (35 kB)
Preparing metadata (setup.py) ... error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-gzd07xzq/napalm-base_aace1b03ac0e4045bbc85e27c788ebc1/setup.py", line 5, in <module>
from pip.req import parse_requirements
ModuleNotFoundError: No module named 'pip.req'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
TL;TR: The problem is that pip version 10, changed the structure for req.
There are several solutions that can be done to make it work, but the easiest way is to downgrade pip, and install the package. After pip can be upgraded again.
Recently a colleague was asking me for advice on their design of error handling in a Python application.
They were catching an error and raising an Exception, inside the except part of a method, to be catch outside the method.
And at some point a simple logic got really messy and unnecessarily complicated. Also troubleshooting and debugging an error was painful because they were only getting a Custom Exception and not context.
I explained to my colleague that I believed that the person that created that Exception chain of catch came from Java background and why I think they choose that path, and why I think in Python it’s a bad idea.
In Java, functions and methods can only return one object.
I programmed a lot in Java in my career, and it was a pain having to create value objects, and having to create all kind of objects for the return. Is it a good thing that types are strongly verified by the language? Yes. It worked? Yes. It made me invest much more time than necessary? Also yes.
Having the possibility to return only one object makes it mandatory having a way to return when there was an error. Otherwise you would need to encapsulate an error code and error description fields in each object, which is contrary to the nature of the object.
For example, a class Persona. Doesn’t make any sense having an attribute inside the class Persona to register if an operation related to this object went wrong.
For example, if we are in a class Spaceship that has a method GetPersonaInCommand() and there is a problem in that method, doesn’t make any sense to return an empty Persona object with attributes idError, errorDescription. Probably the Constructor or Persona will require at least a name or Id to build the object…. so in this case, makes sense that the method raises an Exception so the calling code catches it and knows that something went wrong or when there is no data to return.
This will force to write Custom Exceptions, but it’s a solution.
Another solution is creating a generic response object which could be an Object with these attributes:
an Object which is the response, in our example Persona or null
I created this kind of approach for my Cassandra libraries to easily work with Cassandra from Java and from PHP, and for Cassandra Universal Driver (a http/s gateway created in year 2014).
Why this in not necessary in Python
Python allows you to return multiple values, so I encourage you tor return a boolean for indicating the success of the operation, and the object/value you’re interested.
You can see it easily if you take a look to FileUtils class from my OpenSource libraries carleslibs.
The method get_file_size_in_bytes(self, s_file) for example:
def get_file_size_in_bytes(self, s_file):
b_success = False
i_file_size = 0
# This will help with Unit Testing by raisin IOError Exception
i_file_size = os.path.getsize(s_file)
b_success = True
b_success = False
return b_success, i_file_size
It will always return a boolean value to indicate success or failure of the operation and an integer for the size of the file.
The calling code will do something like this:
o_fileutils = FileUtils()
b_success, i_bytes = o_fileutils.get_file_size_in_bytes("profile.png")
if b_succes is False:
print("Error! The file does not exist or cannot be accessed!")
if i_bytes < 1024:
print("The profile picture should be at least 1KB")
print("Profile picture exists and is", i_bytes, " bytes in length!")
The fact that Python can return multiple variables makes super easy dealing with error handling without having to take the road of Custom Exceptions.
And it is Ok if you want to follow this path, but in my opinion, for most of the developers up to Senior levels, it only over complicates the logic of your code and the amount of try/excepts you have to have everywhere.
If you use PHP you can mix different types in an Array, so you can always return an Array with a boolean, or an i_id_error, and your object or data of whatever type it’s.
Getting back to my carleslibs Open Source package, it is super easy to Unit Test these methods.
In my opinion, this level of simplicity, brings only advantages. Including Software Development speed, which is good for the business.
I’m not advocating for not using Custom Exceptions or to not develop a Exceptions Raising strategy if you need it and you know what you’re doing. I’m just suggesting why I think most of the developments in Python do not really need this and only over complicates the development. There are situations where raising exceptions will be a perfectly useful or even the best approach, there are many scenarios, but I think that in most of cases, using raise inside except will only multiply the time of the development and slow down the speed of bringing new features to the business, over complicating Unit Test as well, and be a real pain for the Junior and Intermediate developers.
Obviously, as the Constructor doesn’t return any value, it is perfectly fine to raise an exception in there, or just to use try/except in the code that is instancing the objects.
This is a document previous to a live code review session.
It has the information to prepare for the upcoming code review session, where I plan to share the lessons learned, decision I took, mistakes I did, refactors I had to overcome, and tentatively we will refactor code in order to add some Unit Testing.
I used to play sudoku with my family, so from time to time I do by myself.
Once I found a sudoku that was impossible and it happened that it was a typo from the newspaper, so, when I found another impossible sudoku I wanted to know if it was me, or if there was a typo or similar, so I decided to write a Sudoku Solver that will solve the sudoku for me.
Tomorrow 23th of April is Sant Jordi (Saint George), the patron of Catalonia, and the Catalan traditional celebration consist in this day women gifting a book to the men they love and men gifting a rose to the women they love.
I created a voucher for all my Python books, so, since during 23th of April you can acquire the four books per $10 USD in total.
So, I was working on a project and i wanted to test how long a file can be.
The resources I checked, and also the Kernel and C source I reviewed, were pointing that effectively the limit is 255 characters.
But I had one doubt… given that Python 3 String are UTF-8, and that Linux encode by default is UTF-8, will I be able to use 255 characters length, or this will be reduced as the Strings will be encoded as UTF-8 or Unicode?.
So I did a small Proof of Concept.
For the filenames I grabbed a fragment of the translation to English of the book epic book “Tirant lo Blanc” (The White Knight), one of the first books written in Catalan language and the first chivalry novel known in the world. You can download it for free it in:
from carleslibs.fileutils import FileUtils
from colorama import Fore, Back, Style , init
# For Colorama on Windows
def print_error(self, s_text, s_end="\n"):
Prints errors in Red.
print(Fore.RED + s_text)
def print_success(self, s_text, s_end="\n"):
Prints errors in Green.
print(Fore.GREEN + s_text)
def print_label(self, s_text, s_end="\n"):
Prints a label and not the end line
print(Fore.BLUE + s_text, end="")
def return_text_blue(self, s_text):
Restuns a Text
s_text_return = Fore.BLUE + s_text + Style.RESET_ALL
if __name__ == "__main__":
o_color = ColorUtils()
o_file = FileUtils()
s_text = "In the fertile, rich and lovely island of England there lived a most valiant knight, noble by his lineage and much more for his "
s_text += "courage. In his great wisdom and ingenuity he had served the profession of chivalry for many years and with a great deal of honor, "
s_text += "and his fame was widely known throughout the world. His name was Count William of Warwick. This was a very strong knight "
s_text += "who, in his virile youth, had practiced the use of arms, following wars on sea as well as land, and he had brought many "
s_text += "battles to a successful conclusion."
o_color.print_label("Erasure Code project by Carles Mateo")
print("Task 237 - Proof of Concep of Long File Names, encoded is ASCii, in Linux ext3 and ext4 Filesystems")
print("Using as a sample a text based on the translation of Tirant Lo Blanc")
print("Sample text used:")
print("-" * 30)
print("This test uses the OpenSource libraries from Carles Mateo carleslibs")
s_dir = "task237_tests"
if o_file.folder_exists(s_dir) is True:
o_color.print_success("Directory " + s_dir + " already existed, skipping creation")
b_success = o_file.create_folder(s_dir)
if b_success is True:
o_color.print_success("Directory " + s_dir + " created successfully")
o_color.print_error("Directory " + s_dir + " creation failed")
for i_length in range(200, 512, 1):
s_filename = s_dir + "/" + s_text[0:i_length]
b_success = o_file.write(s_file=s_filename, s_text=s_text)
s_output = "Writing file length: "
print(" file name: ", end="")
print(": ", end="")
if b_success is False:
# Note: up to 255 work, 256 fails
I tried this in an Ubuntu Virtual Box VM.
As part of my tests I tried to add a non typical character in English, ASCii >127, like Ç.
When I use ASCii character < 128 (0 to 127) for the filenames in ext3/ext4 and in a ZFS Pool, I can use 255 positions. But when I add characters that are not typical, like the Catalan ç Ç or accents Àí the space available is reduced, suggesting the characters are being encoded:
Ç has value 128 in the ASCii table and ç 135.
I used my Open Source carleslibs and the package colorama.
I’m Catalan. In 1936 the fascist military leaded by franco raised in arms against the elected government of the Spanish Republic. The Italian and nazi German fascist in power bombed the Catalan population. Hundreds of thousands of innocent citizens were assassinated and millions of Catalan and Spaniards had to exile. The sons of those that were ruling with the dictator have been insisting in naming it a “civil war”, but it was the military lead by a fascist, revolting against the legitimate Republic and ending a democracy.
The dictatorship lasted until 1975, when the dictator died in the bed. The effects of the repression never abandoned Catalonia, and nowadays in Catalonia people is still detained by the Spanish police for talking the Catalan language in front of them, and our Parliament decisions are cancelled by the Spanish courts, for example to force the exit of a President of Catalonia that they didn’t like, or to force the Catalan schools to teach 25% of the time in Spanish attacking the Catalan teaching system.
During WW2 millions of Jews were mass murdered, also people from all the nations were assassinated.
Russian population suffered a lot also fighting the nazis.
Now we have to see how Russia’s army is invading Ukraine and murdering innocent citizens.
I know Engineers from Ukraine. Those guys were doing great building wealthy based on knowledge and working well for companies across the world. Now these people are being killed or Engineers, amongst all the brave population, are arming themselves to fight the invasion. Shells destroy beautiful cities and population are starving, and young soldiers from both sides will never be seen again by their mothers.
Let music play in solidarity with Ukraine. First is a Catalan group. Second is a famous Irish band in this epic song dedicated to the brave International Brigades, volunteers that fought the fascism in Spain and in Catalonia trying to make a better world.
I’ve updated the SSL Certificate. The previous one I bought was issued for two years, and I renewed as it was due to expire.
Honestly, my ego was flattered. It is a lot of reputation.
Although in the past I got an offer from another monstrously big editorial to publish world wide my book Python 3 Combat Guide and I also rejected, and an offer from a digital learning platform to create an interactive course from this same book.
I’ve rejected it again this time.
If you are curious, this is what I answered to them:
I'm well, thank you. I hope you are doing well too.
Thanks for taking the time to explain your conditions to me.
I feel flattered by your editorial thinking about me. I respect your brand, as I mentioned, as I own several of your titles.
However, I have to refuse your offer.
Is not the first time an editor has offered to publish one or more of my books. For all over the world, with much higher economic expectations.
I'll tell you why I love being at LeanPub:
1- I own the rights. All of them.
2- I can publish updates, and my readers get them for free. As I add new materials, the value is maximized for my readers.
3- I get 80% of the royalties.
4- If a reader is not happy, they can return the book for 60 days.
5- I can create vouchers and give a discount to certain readers, or give for free to people that are poor and are trying to get a career in Engineering.
The community of readers are very honest, and I only got 2 returns. One of them I think was from an editorial that purchased the book, evaluated it, and they contacted me to publish it, and after I rejected they applied for the refund.
I teach classes, and I charge 125 EUR per hour. I can make much more by my side than the one time payment you offer. The compensation for the video seems really obsolete.
Also, I could be using Amazon self publishing, which also brings bigger margins than you.
So many thanks for your offer. I thought about it because of the reputation, but I already have a reputation. I've thousands of visits to my tech blog, and because of the higher royalties, even if I sell less books through LeanPub it is much more rewarding.
Thanks again and have a lovely day and rest of the week.
The provisioning in Amazon AWS through their SDK is a book I’m particularly proud, as it empowers the developers so much. And I provide source code so they can go from zero to hero, in a moment. Amazon should provide a project sample as I do, not difficult to follow documentation.
Teaching / Mentoring
As I was requested, I’ve been offering advice and having virtual coffees with some people that recently started their journey to become Software Engineers and wanted some guidance and advice.
It has been great seeing people putting passion and studying hard to make a better future for themselves and for their families.
I’ll probably add to the blog more contents for beginners, although it will continue being a blog dedicated to extreme IT, and to super cool Engineering skills and troubleshooting.
For my regular students I have a discord space where we can talk and they can meet new friends studying or working in Engineering.
This github link provides many free books in multiple languages:
Zoom can zoom the view. So if they are sharing their screen, and font is too small, you can give a relax to your eyes by using Zoom’s zoom feature. It is located in View.
After being in the hospital in December 2021, with risk for my life, and after my incredible recuperation, I’ve got the good news that I don’t need anymore 2 of the 3 medicines I was taking in a daily basis. It looks well through a completely recovery thanks to my discipline, doing sport every day several times, and the fantastic Catalan doctors that are supporting me so well.
Since they found what was failing in me, and after the emergency treatments I started to sleep really well. All night. That’s a privilege that I didn’t have for long long time.
This is a very simple guide for beginners, to benefit from using this amazing Python IDE with Git.
You may think that it’s a bit difficult and why you should use it. The first reason I would tell is because it marks the errors, not just syntax errors also if you have typos or you forgot an import or you forgot to declare a variable, has autocomplete, will mark when you don’t follow standards, etc… If you want to be a professional, this is the tool that most Python professional developers and companies use.
Go to Jetbrains page and from there select Developer Tools and PyCharm.
If you don’t have Git installed, PyCharm will tell you in red, and will allow you to install it for you. It is a very convenient way to install it, specially if you use Windows.
In Ubuntu or Debian Linux installing git is so easy as typing from the Terminal:
sudo apt install git
If it’s your first time PyCharm will ask you for your credentials in Gitlab, so your username or email, and your password.
As it is you own new project and it’s empty, you can trust it. Hit Trust Project.
A window with tips will appear, you can close it.
As you see your project is empty. Only has a the default README.md file created by Gitlab.
Creating new Python files
On the top left, over the name of your project folder press with the Right button of the mouse and select New > Python File
Enter the name of the file, I choose helloworld and press Enter.
You will be asked if you want to add automatically the file to Git. This refers to your local Git repository. Say Yes and you will see the file changing from color Red, which means it is not added to the repository to Green. Green means that is added, but has never been committed.
Just add this line to helloworld.py
Note: I made a typo and I wrote “helloworl.py” instead of “helloworld.py”, I will show how to fix this later.
Now, press over the file name over the top left, with the Right button of the mouse, and select Run helloworl (In your case will be Run helloword)
Note: To be able to run a Python program, you need to have Python installed.
Note2: Do not start the name of your file by test as PyCharm will consider that you are writing Unit Testing Code, and will not show the option to run the code, but to run with pytest (if you have pytest framework installed).
And you see on the bottom how it worked and printed “Hello World”.
To rename the file you’ll click over it with the Right button of the mouse and select Refactor > Rename
Type the correct name, in my case “helloworld.py” and for the Scope select: Current File.
PyCharm is so powerful that can search for your entire project to see if other files are using that file and will update the code to reflect the file name change. By indicating Scope: Current File we are telling that the change affects only to this file, and by not marking Search for references and not marking Search in comments and strings you are indicating to PyCharm that it should not search in code and comments to update them with the new file. This option is really useful when you get more confident with programming.
Perfect. Now you want to commit the file to the repository and Push.
The first time PyCharm will ask you how people will know you, that is, your visible name in the Git History of Commits, and your email. You can mark to use these properties locally. This is basically because Git needs to know how to identify you. Consider that Git was created to work with Teams, so everybody should know who made what. Add your friendly name, like “Carles Mateo” and your email. I recommend you to check to use this info for Git globally (all the projects in your computer).
When you commit you’re expected to provide a Commit Message. As Git is used by teams, to work on the same code, try to explain briefly what your changes consisted on.
If you press Commit, you will see how helloworld.py is no longer in color green, now is black. This means that the file is up to date respect our local Git.
Go to the Main Menu and select Git > Push.
You’ll see a message in the bottom indicating that your code has been pushed to the repository.
Now, you can refresh your browser, and you’ll see the changes in Gitlab:
PyCharm vs VSCode
I got asked what are the advantage from PyCharm respect VSCode.
I may refer to this excellent comparison:
While VSCode has some great support for Python coding with the ‘Python’ plugin by Microsoft, PyCharm is truly designed for Python development and it shows.
The Unix Epoch is the time, in seconds, that has passed since 1970-01-01 00:00:00 UTC.
# Date time methods
# Author: Carles Mateo
# Creation Date: 2019-11-20 17:23 IST
# Description: Class to return Date, datetime, Unix EPOCH timestamp
Will return the EPOCH Time. For convenience is returned as String
s_now_epoch = str(int(time.time()))
def get_datetime(self, b_milliseconds=False):
Return the datetime with miliseconds in format YYYY-MM-DD HH:MM:SS.xxxxx
or without milliseconds as YYYY-MM-DD HH:MM:SS"""
if b_milliseconds is True:
s_now = str(datetime.datetime.now())
s_now = str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
o_datetime = DateTimeUtils()
s_now_epoch = o_datetime.get_unix_epoch()
s_now_date = o_datetime.get_datetime()
zpool watch is a small Python program for Linux workstations with graphical environment and ZFS, that checks every 30 seconds if your OpenZFS pools are Ok.
If a pool is not healthy, it displays a message in a window using tk inter.
Basically allows you to skip checking from the terminal zpool status continuously or to having to customize the ZED service to send an email and having to figure out how to it can spawn a window alert to the graphical system or what to do if the session has not been initiated.
Since last News from the Blog I’ve released carleslibs v.1.06, v.1.0.5 and v.1.0.4.
v.1.0.6 adds a new class OsUtils to deal with mostly-Linux Os tasks, like knowing the userid, the username, if it’s root, the distribution name and kernel version.
It also adds:
In v.1.0.5 I’ve included a new method for getting the Datetime in Unix Epoc format as Integer and increased Code Coverage to 95% for ScreenUtils class.
v. 1.0.4 contains a minor update, a method in StringUtils to escape html from a string.
It uses the library html (part of Python core) so it was small work to do for me to create this method, and the Unit Test for it, but I wanted to use carleslibs in more projects and adding it as core functionality, makes the code of these projects I’m working on, much more clear.
Minor refactors and adding more Code Coverage (Unit Testing), and protection in the code for division per zero when seconds passed as int are 0. (this was not an actual error, but is worth protecting the code just in case for the future)
Added 20 new pages with some tricks, like clearing the logs (1.6GB in my workstation), using some cool tools, using bind mounts and using Docker in Windows from command line without activating Docker Desktop or WSL.
One of my SATA 2TB 2.5″ 5,400 rpm drive got damaged and so was generating errors, so that was a fantastic opportunity to show how to detect and deal with the situation to replace it with a new SATA 2TB 3.5″ 7,200 rpm and fix the pool.
The company sent me the Stein, which is sent to the employees that serve for two years, with a recognition and a celebration called “The Circle of Honor”.
I bought this book as often I discover new ways, better, to explain the things to my students.
Sometimes I buy books for beginners, as I can get explained what I want to do super fast and some times they teach nice tricks that I didn’t know. I have huge Django books, and it took a lot to finish them.
A simpler book may only talk about how to install and work with it under a platform (Windows or Mac, as instance) but it is all that I require as the command to create projects are the same cross platform.
For example, you can get to install and to create a simple project with ORM, connected to the database, very quickly.
So I just discovered that Zoom has an option to draw in the shared screen, like Slack has. It is called Annotate. It is super useful for my classes. :)
Also discovered the icons in the Chat. It seems that not all the video calls accept it.
As Working From Home I needed an scanner, I looked in Amazon and all of them were costing more than €200.
I changed my strategy and I bought a All-In-One from HP, which costed me €68.
So I’ll have a scanner and a backup printer, which always comes handy.
The nightmare started after I tried to connect it with Ubuntu.
Ubuntu was not recognizing it. Checking the manuals they force to configure the printer from an Android/iPhone app or from their web page, my understanding is for windows only. In any case I would not install the proprietary drivers in my Linux system.
Annoyed, I installed the Android application, and it was requesting to get Location permissions to configure it. No way. There was not possible to configure the printer without giving GPS/Location permissions to the app, so I cancelled the process.
I grabbed a Windows 10 laptop and plugged the All-in-one through the USB. I ran the wizard to search for Scanners and Printers and was not unable to use my scanner, only to configure as a printer, so I was forced to install HP drivers.
Irritated I did, and they were suggesting to configure the printer so I can print from Internet or from the phone. Thanks HP, you’ll be the next SolarWinds big-security-hole. I said no way, and in order to use the Wifi I have to agree to open that security door which is that the printer would be connected to Internet permanently, sending and receiving information. I said no, I’ll use only via USB.
Even selecting that, in order to scan, the Software forces me to create an account.
Disappointing. HP is doing very big stupid mistakes. They used to be a good company.
Since they stopped doing the drivers in Barcelona years ago, their Software and solutions (not the hardware) went to hell.
I checked the reviews in the App Store and so many people gave them 1 star and have problems… what a shame the way they created this solution.
This is a great Open Source, multi-platform editor, so I wanted to support the creator.
Attacks: looking for exploits
This is just a sample of a set of attacks to the blog in a 3 minutes interval.
Another one this morning:
Now all are blocked in the Firewall.
This is a non stop practice from spammers and pirates that has been going on for years.
It was almost three decades ago, when I was the Linux responsible of an ISP, and I was installing a brand new Linux system connected to a service called “infovia”, at the time when Internet was used with dial-up and modems, and in the interval of time of the installation, it got hacked. I had the Ethernet connected. So then already, this was happening.
The morning I was writing this, I blocked thousands of offending Ip Addresses.
I recommend you to use CloudFlare, is a CDN/Cache/Accelerator with DoS protection and even in its Free version is really useful.
So I come with a game kind of Quiz that you can play with your friends, family or work colleagues working from home (WFH).
The idea is that the master shares screen and sound in Zoom, and then the rest connect to jackbox.tv and enter the code displayed on the master’s screen on their own browser, and an interactive game is started.
It is recommended that the master has two monitors so they can also play.
The games are so fun as a phrase appearing and people having to complete with a lie. If your friends vote your phrase, believing is true, you get points. If you vote the true answer, you get points too.
Very funny and recommendable.
<humor>Skynet sent another terminator to end me, but I terminated it. Its processor lays exhibited in my home now</humor>
I bought a laminator.
It has also a ruler and a trimmer to cut the paper.
It was only €39 and I’ve to say that I’m very happy with the results.
It takes around 5 minutes to be ready, it takes to get to the hot-enough temperature, and feeds the pages slowly, around 50 secs a DIN-A4, but the results are worth the time.
I’ve protected my medical receipts and other value documents and the work was perfect. No bubbles at all. No big deal if the plastic covers are introduced not 100% straight. Even if you pass again an already plasticized document, all is good.
One of my friends sent me this image.
It is old, but still it’s fun. So it assumes the cameras of the parking or speed cameras, will OCR the plate to build a query, and that the code is not well protected. So basically is exploiting a Sql Injection.
Anybody working on the systems side, and with databases, knows how annoying are those potential situations.