Tag Archives: Python 3

Web Top – Displaying top with Python 3 Web Server and Carleslibs

So this is a super simple example on how quickly you can create nice solutions with my package carleslibs.

In this example I use Python 3 incorporated Web Server and carleslibs, to execute top and display in the browser.

Requisites:

Having Python3 and have installed carleslibs 1.0.1 or superior.

pip3 install carleslibs

Having this running in a Linux with top installed. All of them come with top, as long as I know.

This is the 84 lines code for WebTop:

from http.server import BaseHTTPRequestHandler, HTTPServer

from carleslibs.subprocessutils import SubProcessUtils
from carleslibs.datetimeutils import DateTimeUtils


class Top():

    def __init__(self, o_subprocess):
        self.o_subprocess = o_subprocess

    def get_top(self):

        a_domains_offline = []

        s_command = "/usr/bin/top -n 1 -b"
        i_code, s_stdout, s_stderr = self.o_subprocess.execute_command_for_output(s_command, b_shell=True, b_convert_to_ascii=True)

        return i_code, s_stdout, s_stderr


class WebServer(BaseHTTPRequestHandler):

    def do_GET(self):
        o_subprocess = SubProcessUtils()
        self.o_top = Top(o_subprocess)

        self.o_datetime = DateTimeUtils()

        self.i_max_domains_offline = 0

        self.send_response(200)
        self.send_header("Content-type", "text/html")
        self.end_headers()

        WebTop.log(self.path)
        if self.path == "/favicon.ico":
            return

        s_html = "<html><body>"
        s_html = s_html + "<h1>Web Top</h1>"
        s_html = s_html + '<small>by Carles Mateo - <a href="https://blog.carlesmateo.com">blog.carlesmateo.com</a></small>'

        i_code, s_stdout, s_stderr = self.o_top.get_top()

        if i_code != 0:
            s_html = s_html + "Error Code: " + str(i_code) + "&lt;/br&gt;"
            s_html = s_html + "Message: " + s_stderr + "&lt;/br&gt;"
        else:
            s_html = s_html + "<pre>"
            s_html = s_html + s_stdout
            s_html = s_html + "</pre>"

        s_html = s_html + "</body>"
        s_html = s_html + "</html>"

        by_html = bytes(s_html, encoding="utf-8")

        self.wfile.write(by_html)


class WebTop():

    o_datetime = DateTimeUtils()

    @staticmethod
    def log(s_text):
        s_datetime = WebTop.o_datetime.get_datetime()
        print(s_datetime, s_text)


if __name__ == "__main__":

    o_webserver = HTTPServer(("localhost", 80), WebServer)
    WebTop.log("Server started")

    try:
        o_webserver.serve_forever()
    except KeyboardInterrupt:
        pass

    o_webserver.server_close()
    WebTop.log("Server stopped")

Just run the code and go to localhost with your favorite browser.

If you get an error like this it means that another process is listening on port 80. Just use another like 8080, 8181, etc…

Traceback (most recent call last):
  File "/home/carles/Desktop/code/carles/json-realm-live/web_top.py", line 74, in <module>
    o_webserver = HTTPServer(("localhost", 80), WebServer)
  File "/usr/lib/python3.8/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/usr/lib/python3.8/http/server.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/usr/lib/python3.8/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied

And you will see the requests coming:

2021-10-09 10:47:46 Server started
127.0.0.1 - - [13/Oct/2021 10:47:48] "GET / HTTP/1.1" 200 -
2021-10-09 10:47:48 /
127.0.0.1 - - [13/Oct/2021 11:25:24] "GET / HTTP/1.1" 200 -
2021-10-09 11:25:24 /

So instead of using top, you can use ctop.py :)

Just replace the command by:

s_command = "ctop.py -b -n=1 --rows=30 --columns=200"

You can also create a Dockerfile very easily and run this in a Container.

Some graphics with matplotlib

Recently I showed you how to generate a Cloud Tag.

You may like some of other graphs that can be easily generated with matlib package.

I’ve been always working on BackEnd and APIs and I don’t work on FrontEnd, although I programmed some videogames by myself and I’ve fixed some huge bugs in JavaScript in some of the companies I work, but they considered myself the last resource, so I would fix a FrontEnd bug when nobody else could. But even if you work 99.9% of your time in BackEnd, Scaling, Architecture… like me, it is useful being able to draw graphics, for example, when you create a tool that shows the number of players per minute, and its evolution over time, or web visitors in real time, etc…

I wrote this article with two simple examples for my book Python 3 exercises for beginners.

You can find this source code here:

https://gitlab.com/carles.mateo/python-classes/-/blob/main/2021-09-10/draw_points.py


import matplotlib.pyplot as plt

a_points1 = [7, 3, 15, 5, 10, 2, 9]
a_points2 = [2, 4, 9, 2, 7, 8, 4]

plt.plot(a_points1)
plt.plot(a_points2)

plt.show()

We can also add customized axis:

https://gitlab.com/carles.mateo/python-classes/-/blob/main/2021-09-10/draw_points2.py

import matplotlib.pyplot as plt

a_points1 = [7, 3, 15, 5, 10, 2, 9]
a_points2 = [2, 4, 9, 2, 7, 8, 4]
a_points3 = [12, 10, 1, 7, 14, 16, 1]

a_days_of_the_week = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"]

plt.plot(a_days_of_the_week, a_points1)
plt.plot(a_days_of_the_week, a_points2)
plt.plot(a_days_of_the_week, a_points3)
plt.grid(axis='y', color='black', linestyle='solid')
plt.show()

Draw a pie chart

import matplotlib.pyplot as plt

a_scores = [70, 20, 5, 5]
a_languages = ["Python", "Bash", "Java", "PHP"]
a_colors = ["Red", "Blue", "Green", "Cyan"]

plt.pie(a_scores, labels=a_languages, colors=a_colors)
plt.legend()
plt.show()

This graphic represents in which languages I use my time nowadays, or if I update it by adding HTML and jQuery:

https://gitlab.com/carles.mateo/python-classes/-/blob/main/2021-09-10/draw_circle.py

Some weird things from Python 3 that you may not know

Last Update: 2022-06-06 10:29 IST

You can find those bizarre things and more in my book Python 3 Combat Guide.

I’m not talking about the wonderful things, like how big can the Integers be, but about the bizarre things that may ruin your day.

What sums 0.1 + 0.1 + 0.1 in Python?

0.3?

Wrong answer.

A bit of humor

Well, to be honest the computer was wrong. They way programming languages handle the Floats tend to be less than ideal.

Floats

Maybe you know JavaScript and its famous NaN (Not a number).

You are probably sure that Python is much more exact than that…

…well, until you do a big operation with Floats, like:

10.0**304 * 10.0**50 

and

It returns infinite

I see your infinite and I add one :)

However If we try to define a number too big directly it will return OverflowError:

Please note Integers are handled in a much more robust cooler way:

Negative floats

Ok. What happens if we define a number with a negative power, like 10 ** -300 ?

And if we go somewhere a bit more far? Like 10 ** -329

It returns 0.0

Ups!

I mention in my books why is better to work with Integers, and in fact most of the eCommerces, banks and APIs work with Integers. For example, if the amount in USD 10.00 they send multiplied by 100, so they will send 1000. All the actor know that they have to divide by 2.

Breaking the language innocently

I mentioned always that I use the MT Notation, the prefix notation I invented, inspired by the Hungarian Notation and by an amazing C++ programmer I worked with in Volkswagen and in la caixa (now caixabank), that passed away many years ago.

Well, that system of prefixes will name a variable with a prefix for its type.

It’s very useful and also prevents the next weird thing from Python.

Imagine a Junior wants to print a String and they put in a variable. And unfortunately they call this variable print. Well…

print = "Hello World!"
print("That will hurt")

Observe the output of this and try not to scream:

Variables and Functions named equally

Well, most of languages are able to differentiate a function, with its parenthesis, from a variable.

The way Python does it hurts my coder heart:

Another good reason to use MT Notation for the variables, and for taking seriously doing Unit Testing and giving a chance to using getters and setters and class Constructor for implementing limits and sanitation.

Nested Loops

This will work in Python, it doesn’t work in other languages (but please never do it).

for i in range(3):
    print("First Loop", i)
    for i in range(4):
        print("Second Loop", i)

The code will not crash by overwriting i used in the first loop, but the new i will mask the first variable.

And please, name variables properly.

Import… once?

Imports are imported only once. Even if different files imported do import the same file.

So don’t have code in the middle of them, outside functions/classes, unless you’re really know what you’re doing.

Define functions first, and execute code after if __name__ == “__main__”:

Take a look at this code:

def first_function():
    print("Inside first function")
    second_function()

first_function()

def second_function():
    print("Inside second function")

Well, this will crash as Python executes the code from top to bottom, and when it gets to first_function() it will attempt to call second_function() which has not been read by Python yet. This example will throw an error.

You’ll get an error like:

Inside first function
Traceback (most recent call last):
  File "/home/carles/Desktop/code/carles/python_combat_guide/src/structure_dont_do_this.py", line 14, in <module>
    first_function()
  File "/home/carles/Desktop/code/carles/python_combat_guide/src/structure_dont_do_this.py", line 12, in first_function
    second_function()
NameError: name 'second_function' is not defined

Process finished with exit code 1

Add your code at the bottom always, under:

if __name__ == "__main__":
    first_function()

The code inside this if will only be executed if you directly call this code as main file, but will not be executed if you import this file from another one.

You don’t have this problem with classes in Python, as they are defined first, completely read, and then you instantiate or use them. To avoid messing and creating bugs, have the imports always on the top of your file.

…Ellipsis

Today is Halloween and one of my colleagues asked me help to improve his Automation project.

I found something weird in his code.

He had something like that.

class Router:

    def router_get_info(self):
        ...

    def get_help_command(self):
        return "help"

So I asked why you use … (dot dot dot) on that empty method?.

He told me that when he don’t want to implement code he just put that.

Well, dot dot dot is Ellipsis.

And what is Ellipsis?.

Ellipsis is an object that may appear in slice notation.

A good explanation of what is Ellipsis is found in this answer in StackOverflow.

In Python all the methods, functions, if, while …. require to have an instruction at least.

So the instruction my colleague was looking for is pass.

Just a variable?

In Python you can have just a var, without anything else, like no operation with it, no call, nothing.

This makes it easy to commit an error and not detecting it.

As you see we can have just s_var variable in a line, which is a String, and this does not raises an error.

If we do from python interpreter interactively, it will print the String “I’m a pickle” (famous phrase from Rick and Morty).

Variables are case sensitive

So you can define true false none … as they are different from True False None

Variables in Unicode

Python3 accepts variables in Unicode.

I would completely discourage you to use variables with accents or other characters different from a-z 0-9 and _

Python files with these names yes, but kaboom if you import them

So you can create Python files with dash or beginning with numbers, like 20220314_programming_class.py and execute them, but you cannot import them.

RYYFTK RODRIGUEZ,LEELA,FRY, FUTURAMA, 1999

A Tuple of a String is not a Tuple, it’s a String

This can be very messy and confusing. Normally you define a tuple with parenthesis, although you can use tuple() too.

Parenthesis are the way we normally build tuples. But if we do:

print(type('this is a String'))

You get that this is a String, I mean

<class 'str'>

If you want to get a tuple of a String you can add a comma after the first String, which is weird. You can also do tuple("this is a String")

I think the definition of a tuple should be consistent and idempotent, no matter if you use one or more parameters. Probably as parenthesis are used for other tasks, like invoking functions or methods, or separating arithmetic operations, that reuse of the signs () for multiple purposes is what caused a different behavior depending on if there is one or more parameters the mayhem IMO.

See some example cases.

Python simplifies the jump of line \n platform independent and some times it’s messy

If you come from a C background you will expect text file in different platforms: Linux, Mac OS X (changes from old to new versions), Windows… to be represented different. In some cases this is an ASCii code 10 (LF), in others 13 (CR), and in other two characters: 13 and immediately after 10.

Python simplifies the Enter character by naming it \n like in C.

So, platform independent, whenever you read a text file you will get \n for any ASCii 10 [LF] or 13 [CR]. [CR] will be converted to [10] in Linux.

If you read a file in a Linux system, where enters are represented by 10, which was generated in a Windows system, so it has [CR][LF] instead of [LF] at the end of each line, you’ll get a \n too, but two times.

And if you do len(“\n”) to know the len of that String, this returns 1 in all the platform.

To read the [LF] and [CR] (represented by \r) you need to open the file as binary. By default Python opens the files as text.

You can check this by writting [LF] and [CR] in Linux and see how Python seamlessly reads the file as it was [LF].

A file generated by Windows will get \n\n:

Random code when the class is imported

In a procedural file, the code that is outside a function, will be executed when it is imported. But if this file is imported again it will not be re-executed.

Things are more messy if you import a class file. Inside the body of the class, in the space you would reserve for static variables definition, you can have random code. And this code will be only executed on the first import, not on subsequent.

Disclaimer: the pictures from Futurama are from their respective owners.

Generating a Word Cloud of Tags in Python

This is a very simple code but generates very cool Word Cloud result in PNG format.

from wordcloud import WordCloud

# Add your tags in here separated by commas and as many times as they appear
s_text = "Python, Software development, PHP, Cloud providers, Python, Python, Software development, Scaling"

o_word_cloud = WordCloud(height=800,
                         width=800,
                         background_color="white",
                         max_words=150,
                         min_font_size=5,
                         collocation_threshold=10).generate_from_text(s_text)

o_word_cloud.to_file("words.png")

That version generated the image .PNG file.

If you want to display this in mathlib or inside PyCharm embedded view, you can install matplotlib with:

pip3 install matplotlib

Then with this code you can display a matplotlib viewer:

import matplotlib.pyplot as plt
from wordcloud import WordCloud

if __name__ == "__main__":
    # Add your tags in here separated by commas and as many times as they appear
    s_text = "Python, Software development, PHP, Cloud providers, Python, Python, Software development, Scaling"

    o_word_cloud = WordCloud(height=800,
                             width=800,
                             background_color="white",
                             max_words=150,
                             min_font_size=5,
                             collocation_threshold=10).generate_from_text(s_text)

    plt.figure(figsize=(10,8))
    plt.imshow(o_word_cloud)
    plt.axis("off")
    plt.tight_layout(pad=0)
    plt.show()

Python Game Tic Tac Toe

I implemented this very simple game for my book Python 3 Exercises for Beginners.

Source Code available here:

https://gitlab.com/carles.mateo/python-classes/-/blob/main/2021-09-10/game_tic-tac-toe.py


class TicTacToe:

    def __init__(self):
        self.a_a_s_map = []
        self.generate_map()

    def generate_map(self):
        self.a_a_s_map = []

        for i_y in range(3):
            a_s_pos_x = [" ", " ", " "]
            self.a_a_s_map.append(a_s_pos_x)

    def get_map(self):
        s_map = ""

        s_map = s_map + "    1   2   3\n"
        s_map = s_map + "  -------------\n"
        for i_y in range(3):
            s_map = s_map + str(i_y + 1) + " |"
            for s_char in self.a_a_s_map[i_y]:
                s_map = s_map + " " + s_char + " |"
            s_map = s_map + "\n"
            s_map = s_map + "  -------------\n"

        return s_map

    def validate_move(self, s_char, i_x, i_y):
        """
        Validates the movement and updates the map
        :param s_char:
        :param i_x:
        :param i_y:
        :return: bool
        """
        i_x = i_x - 1
        i_y = i_y - 1

        if self.a_a_s_map[i_y][i_x] == " ":
            self.a_a_s_map[i_y][i_x] = s_char
            return True

        return False

    def check_win(self):
        for s_char in ["O", "X"]:

            # check horizontal
            for i_y in range(3):
                i_horizontal_match = 0
                for i_x in range(3):
                    if self.a_a_s_map[i_y][i_x] == s_char:
                        i_horizontal_match = i_horizontal_match + 1
                if i_horizontal_match == 3:
                    return True

            # Check vertical
            for i_x in range(3):
                i_vertical_match = 0
                for i_y in range(3):
                    if self.a_a_s_map[i_y][i_x] == s_char:
                        i_vertical_match = i_vertical_match + 1
                if i_vertical_match == 3:
                    return True

            # Check diagonal
            if self.a_a_s_map[1][1] == s_char:
                if self.a_a_s_map[0][0] == s_char and self.a_a_s_map[2][2] == s_char:
                    return True
                if self.a_a_s_map[0][2] == s_char and self.a_a_s_map[2][0] == s_char:
                    return True

        return False

    def check_stale(self):
        for i_y in range(3):
            for i_x in range(3):
                if self.a_a_s_map[i_y][i_x] == " ":
                    # Is not full
                    return False

        # We checked all and all were full
        return True


def get_from_keyboard(s_question, i_min, i_max):
    i_number = 0
    while True:
        s_answer = input(s_question)
        try:
            i_number = int(s_answer)
        except:
            print("Please, type a number")
            continue

        if i_number < i_min or i_number > i_max:
            print("Invalid value. Values should be between", i_min, "and", i_max)
            continue

        # Validations are Ok
        break

    return i_number


if __name__ == "__main__":
    o_tictactoe = TicTacToe()

    while True:

        s_map = o_tictactoe.get_map()
        print(s_map)

        while True:
            i_x = get_from_keyboard("Your move O for x: ", i_min=1, i_max=3)
            i_y = get_from_keyboard("Your move O for y: ", i_min=1, i_max=3)

            b_valid_move = o_tictactoe.validate_move("O", i_x, i_y)
            if b_valid_move is False:
                print("Invalid move")
                continue

            break

        s_map = o_tictactoe.get_map()
        print(s_map)
        b_check_win = o_tictactoe.check_win()
        if b_check_win is True:
            print("Player O wins!")
            exit(0)

        b_stale = o_tictactoe.check_stale()
        if b_stale is True:
            print("Nobody wins in war")
            exit(0)

        while True:
            i_x = get_from_keyboard("Your move X for x: ", i_min=1, i_max=3)
            i_y = get_from_keyboard("Your move X for y: ", i_min=1, i_max=3)

            b_valid_move = o_tictactoe.validate_move("X", i_x, i_y)
            if b_valid_move is False:
                print("Invalid move")
                continue

            break

        s_map = o_tictactoe.get_map()
        print(s_map)
        b_check_win = o_tictactoe.check_win()
        if b_check_win is True:
            print("Player X wins!")
            exit(0)

A sample Flask application

Today I bring you a game made with Python and Flask extracted from my book Python 3 Combat Guide.

It is a very simple game where you have to choose what Star wars robot you prefer.

Then an internal counter, kept in a static variable, is updated.

I display the time as well, to show the use of a in import and dynamic contents printed as well.

I added a Dockerfile and a bash script to build the Docker Image, so you can run the Docker Container without installing anything in your computer.

You can download the code from here:

https://gitlab.com/carles.mateo/python-flask-r2d2

Or clone the project:

git clone https://gitlab.com/carles.mateo/python-flask-r2d2.git

Then build the image with the script I provided:

sudo ./build_docker.sh 

After Docker Image flask_app is built, you can run a Docker Container based on it with:

sudo docker run -d -p 5000:5000 --name flask_app flask_app

After you’re done, in order to stop the Container type:

sudo docker stop flask_app

Here is the source code of the Python file flask_app.py:

#
# flask_app.py
#
# Author: Carles Mateo
# Creation Date: 2020-05-10 20:50 GMT+1
# Description: A simple Flask Web Application
#              Part of the samples of https://leanpub.com/pythoncombatguide
#              More source code for the book at https://gitlab.com/carles.mateo/python_combat_guide
#

from flask import Flask
import datetime


def get_datetime(b_milliseconds=False):
    """
    Return the datetime with miliseconds in format YYYY-MM-DD HH:MM:SS.xxxxx
    or without milliseconds as YYYY-MM-DD HH:MM:SS
    """
    if b_milliseconds is True:
        s_now = str(datetime.datetime.now())
    else:
        s_now = str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))

    return s_now


app = Flask(__name__)

# Those variables will keep their value as long as Flask is running
i_votes_r2d2 = 0
i_votes_bb8 = 0


@app.route('/')
def page_root():
    s_page = "<html>"
    s_page += "<title>My Web Page!</title>"
    s_page += "<body>"
    s_page += "<h1>Time now is: " + get_datetime() + "</h1>"
    s_page += """<h2>Who is more sexy?</h2>
<a href="r2d2"><img src="static/r2d2.png"></a> <a href="bb8"><img width="250" src="static/bb8.jpg"></a>"""
    s_page += "</body>"
    s_page += "</html>"

    return s_page


@app.route('/bb8')
def page_bb8():
    global i_votes_bb8

    i_votes_bb8 = i_votes_bb8 + 1

    s_page = "<html>"
    s_page += "<title>My Web Page!</title>"
    s_page += "<body>"
    s_page += "<h1>Time now is: " + get_datetime() + "</h1>"
    s_page += """<h2>BB8 Is more sexy!</h2>
                <img width="250" src="static/bb8.jpg">"""
    s_page += "<p>I have: " + str(i_votes_bb8) + "</p>"
    s_page += "</body>"
    s_page += "</html>"

    return s_page


@app.route('/r2d2')
def page_r2d2():
    global i_votes_r2d2

    i_votes_r2d2 = i_votes_r2d2 + 1

    s_page = "<html>"
    s_page += "<title>My Web Page!</title>"
    s_page += "<body>"
    s_page += "<h1>Time now is: " + get_datetime() + "</h1>"
    s_page += """<h2>R2D2 Is more sexy!</h2>
                <img src="static/r2d2.png">"""
    s_page += "<p>I have: " + str(i_votes_r2d2) + "</p>"
    s_page += "</body>"
    s_page += "</html>"

    return s_page


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000, debug=True)

As always, the naming of the variables is based on MT Notation.

The Dockerfile is very straightforward:

FROM ubuntu:20.04

MAINTAINER Carles Mateo

ARG DEBIAN_FRONTEND=noninteractive

RUN apt update && \
    apt install -y vim python3-pip &&  pip3 install pytest && \
    apt-get clean

ENV PYTHON_COMBAT_GUIDE /var/python_combat_guide

RUN mkdir -p $PYTHON_COMBAT_GUIDE

COPY ./ $PYTHON_COMBAT_GUIDE

ENV PYTHONPATH "${PYTHONPATH}:$PYTHON_COMBAT_GUIDE/src/:$PYTHON_COMBAT_GUIDE/src/lib"

RUN pip3 install -r $PYTHON_COMBAT_GUIDE/requirements.txt

# This is important so when executing python3 -m current directory will be added to Syspath
# Is not necessary, as we added to PYTHONPATH
#WORKDIR $PYTHON_COMBAT_GUIDE/src/lib

EXPOSE 5000

# Launch our Flask Application
CMD ["/usr/bin/python3", "/var/python_combat_guide/src/flask_app.py"]

Video: Beginners Python: teaching to work with for, range, lists, dicts

I recorded this class where some university colleagues and few of my students joined me.

It is recorded at Full HD. Increase Youtube’s quality to make sure the video is smoothly displayed.

You can download the source code from a previous class and this one in Gitlab:

https://gitlab.com/carles.mateo/python-classes

If you are looking for my technical books follow the link.

If you would like to become one of my students, or to have me as Mentor, check this section in my blog:

Classes and Mentor

News from the Blog 2021-07-01

  • Google Instances’ Performance
    I’ve updated the CMIPS score for the latest Google instances vs last Amazon’s I tried and baremetals.

This is the changelog for latest version:

v. 0.99
 A whole new chapter showing sorting in Python and lambdas. (.sort() and sorted() package First)

 I show writing lambdas for Sorting, and also what makes them crash.

 Explained why Lambdas are not recommended unless you use for working with data, like for sorting or filtering out, and unless you know what you are doing. They are difficult to Debug.

 Explained about PEP8 tool to validate style.

 Explaining why we define Instance variables in the Constructor.

 Provided more samples for Flask Applications.

 Fixed code sample https://gitlab.com/carles.mateo/python_combat_guide/-/blob/master/src/keywords.py as the editor removed the white line spaces.

 Added more books to the bibliography

 I explain the importance of running Unit Testing as both root and as regular users.

 Explain how to run as regular user inside a Docker Container.

 Explained requirements.txt file. And how integrates PyCharm to create venv/ Virtual Environment.

 Also how it is used in Dockerfile to make sure all the dependencies are satisfied in the Docker Container.

As any project committed to saving human lives, she has all my support and admiration.

A base Dockerfile for my Jenkins deployments

Update: I’ve created a video and article about how to install jenkins in Docker with docker CLI and Blue Ocean plugins following the official Documentation. You may prefer to follow that one.

Update: Second part of this article: Creating Jenkins configurations for your projects

So I share with you my base Jenkins Dockerfile, so you can spawn a new Jenkins for your projects.

The Dockerfile installs Ubuntu 20.04 LTS as base image and add the required packages to run jenkins but also Development and Testing tools to use inside the Container to run Unit Testing on your code, for example. So you don’t need external Servers, for instance.

You will need 3 files:

  • Dockerfile
  • docker_run_jenkins.sh
  • requirements.txt

The requirements.txt file contains your PIP3 dependencies. In my case I only have pytest version 4.6.9 which is the default installed with Ubuntu 20.04, however, this way, I enforce that this and not any posterior version will be installed.

File requirements.txt:

pytest==4.6.9

The file docker_run_jenkins.txt start Jenkins when the Container is run and it will wait until the initial Admin password is generated and then it will display it.

File docker_run_jenkins.sh:

#!/bin/bash

echo "Starting Jenkins..."

service jenkins start

echo "Configure jenkins in http://127.0.0.1:8080"

s_JENKINS_PASSWORD_FILE="/var/lib/jenkins/secrets/initialAdminPassword"

i_PASSWORD_PRINTED=0

while [ true ];
do
    sleep 1
    if [ $i_PASSWORD_PRINTED -eq 1 ];
    then
        # We are nice with multitasking
        sleep 60
        continue
    fi

    if [ ! -f "$s_JENKINS_PASSWORD_FILE" ];
    then
        echo "File $s_FILE_ORIGIN does not exist"
    else
        echo "Password for Admin is:"
        cat $s_JENKINS_PASSWORD_FILE
        i_PASSWORD_PRINTED=1
    fi
done

That file has the objective to show you the default admin password, but you don’t need to do that, you can just start a shell into the Container and check manually by yourself.

However I added it to make it easier for you.

And finally you have the Dockerfile:

FROM ubuntu:20.04

LABEL Author="Carles Mateo" \
      Email="jenkins@carlesmateo.com" \
      MAINTAINER="Carles Mateo"

# Build this file with:
# sudo docker build -f Dockerfile -t jenkins:base .
# Run detached:
# sudo docker run --name jenkins_base -d -p 8080:8080 jenkins:base
# Run seeing the password:
# sudo docker run --name jenkins_base -p 8080:8080 -i -t jenkins:base
# After you CTRL + C you will continue with:
# sudo docker start
# To debug:
# sudo docker run --name jenkins_base -p 8080:8080 -i -t jenkins:base /bin/bash

ARG DEBIAN_FRONTEND=noninteractive

ENV SERVICE jenkins

RUN set -ex

RUN echo "Creating directories and copying code" \
    && mkdir -p /opt/${SERVICE}

COPY requirements.txt \
    docker_run_jenkins.sh \
    /opt/${SERVICE}/

# Java with Ubuntu 20.04 LST is 11, which is compatible with Jenkins.
RUN apt update \
    && apt install -y default-jdk \
    && apt install -y wget curl gnupg2 \
    && apt install -y git \
    && apt install -y python3 python3.8-venv python3-pip \
    && apt install -y python3-dev libsasl2-dev libldap2-dev libssl-dev \
    && apt install -y python3-venv \
    && apt install -y python3-pytest \
    && apt install -y sshpass \
    && wget -qO - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add - \
    && echo "deb http://pkg.jenkins.io/debian-stable binary/" > /etc/apt/sources.list.d/jenkins.list \
    && apt update \
    && apt -y install jenkins \
    && apt-get clean

RUN echo "Setting work directory and listening port"
WORKDIR /opt/${SERVICE}

RUN chmod +x docker_run_jenkins.sh

RUN pip3 install --upgrade pip \
    && pip3 install -r requirements.txt


EXPOSE 8080


ENTRYPOINT ["./docker_run_jenkins.sh"]

Build the Container

docker build -f Dockerfile -t jenkins:base .

Run the Container displaying the password

sudo docker run --name jenkins_base -p 8080:8080 -i -t jenkins:base

You need this password for starting the configuration process through the web.

Visit http://127.0.0.1:8080 to configure Jenkins.

Configure as usual

Resuming after CTRL + C

After you configured it, on the terminal, press CTRL + C.

And continue, detached, by running:

sudo docker start jenkins_base

The image is 1.2GB in size, and will allow you to run Python3, Virtual Environments, Unit Testing with pytest and has Java 11 (not all versions of Java are compatible with Jenkins), use sshpass to access other Servers via SSH with Username and Password…

A live session refactoring and adding Unit Testing to my Python3 project cmemgzip

I refactor and add unit testing to my actual project cmemgzip while I comment every step so you can learn the whys and the reasoning.

Open video in full screen with max 1080 quality, in order to see the code clearly.

Update 2021-03-03: I added a third part. I’m tired but still is worth watching.