Tag Archives: DigitalOcean

News from the blog 2022-07-22

Carles in the News

For all my friends and followers, I started to translate my radio space “El nou món digital” (in Catalan) to English “The New Digital World”. I cover Science, Technology, Entertainment and Video games.

You can see the first programs I translated here:

Catalan and English programs RAB

And the script and links mentioned here:

https://blog.carlesmateo.com/2022/06/20/rab-el-nou-mon-digital-2022-06-27-ca/

Social

I’ve created an Instagram fan page for me / for the blog.

It is open for everybody.

https://www.instagram.com/blog_carlesmateo/

Videos for learning how to code

Learning How to code Python Unit Tests with pytest from the scratch, for beginners:

I added it to my series of videos:

https://blog.carlesmateo.com/learn-python-3-unit-testing/

Video for learning how to use RabbitMQ with Python in a Docker Container

https://blog.carlesmateo.com/2022/07/20/creating-a-rabbitmq-docker-container-accessed-with-python-and-pika/

Site carlesmateo.com

I use mostly this site https://blog.carlesmateo.com to centralize everything, so I’ve kept http://carlesmateo.com as a simple landing page made with my old (from 2013) ultra fast PHP Framework Catalonia Framework.

I decided to create a Jenkins Pipeline to deploy any updates to this pages and I updated it a bit at least to provide the most common information searched.

Don’t expect anything fancy at Front End level. :)

Cloud

I created a video about how to provision a Ubuntu Droplet in Digital Ocean.

It’s just for beginners, or if you used other CSP’s and you wonder how Digital Ocean user interface is.

It is really easy, to be honest. Amazon AWS should learn from them.

I also created another about how to provision using User Data Cloud Init feature:

https://blog.carlesmateo.com/2022/06/25/how-to-deploy-a-digitalocean-droplet-instance-and-use-userdata/

Books

My Books

I have updated Docker Combat Guide to show how to work with different users in the Dockerfile and accessing an interactive terminal.

I have also added how to create a Jenkins containerized.

Books I bought > CI/CD

I recommend you these books:

I’ve created a video about how to deploy jenkins in Docker, following the official documentation, in 4 minutes.

Install jenkins on Docker in ubuntu in 4 minutes

And posted an article about solving the error load key invalid format when provisioning from Jenkins with your SSH .pem Key in a variable.

https://blog.carlesmateo.com/2022/07/05/solving-linux-load-key-ssh_yourserver-invalid-format-when-provisioning-from-jenkins/

Open Source from Carles

CTOP.py

I have released a new version of CTOP, ctop.py version 0.8.9.

This version fixes few bugs, and adds better Unit Testing Code Coverage, and is integrated with jenkins (provides Jenkinsfile and a Dockerfile ready to automate the testing pipeline)

Sudo ku solver, Sudoku solver in Python, and Engineering solving problem approach

I’ve created this video explaining my experience writing a program to solve two impossible, very annoying Sudokus. :)

https://blog.carlesmateo.com/2022/04/26/working-on-a-sudoku-solver-in-python-source-code/

Commander Turtle: a small program in Python so children can learn to code by drawing

Children can learn to code and share their scripts, which are comma separated, easily.

Commander Turtle

My life at Activision Blizzard

We have released World of Warcraft Dragonflight Alpha.

In the sync meetings I lead with Wow SRE and product Team I was informed that streaming would be open. Myself I was granted to stream over twitch, but so far I didn’t want to stream video games in my engineering channels. It’s different kind of audiences IMO. Let me know if you would like to get video game streams in my streaming channels.

Lich King

Humor

If you have been in a madness of Servers of a Cluster getting irresponsible and having to cold reboot them from remote hands iDracs or similar, you know why my friends sent me this image :D

How to deploy a DigitalOcean droplet (instance) and use userdata

You can see how to deploy a new droplet also named Instance or VM to DigitalOcean Cloud Provider in this video.

You can see how User Data works in this other video.

The userdata script is run as root the first time the instance is deployed.

I provisioned using this userdata script:

#!/bin/bash

sudo apt update && sudo apt install -y net-tools vim mc htop less strace zip gzip lynx curl wget git apache2

cd /opt/
git clone https://gitlab.com/carles.mateo/ctop.git

If you look for ctop.py check this page:

CTOP.py

For information about User Data from Digital Ocean:

https://docs.digitalocean.com/products/droplets/how-to/provide-user-data/

News from the blog 2022-04-22

Media/Press

I was interviewed by Radio America Barcelona, in their studios in Barcelona.

RAB is a radio for the Catalan diaspora and expats.

The interview was broadcasted by Twitch and can be watched. It’s in Catalan language:

https://www.twitch.tv/videos/1448895585

You can follow them:

Sant Jordi discounted books (promo)

Tomorrow 23th of April is Sant Jordi (Saint George), the patron of Catalonia, and the Catalan traditional celebration consist in this day women gifting a book to the men they love and men gifting a rose to the women they love.

I created a voucher for all my Python books, so, since during 23th of April you can acquire the four books per $10 USD in total.

This voucher is limited to 100 sales.

https://leanpub.com/b/python3all/c/SANTJORDI2022

Python 3

I wrote an article with a Python 3 code that shows the length for file names in Linux ext3 and ext4 Filesystems and in ZFS Pools Filesystem.

I show basically how ASCii characters over 127 are encoded, reducing the maximum length of 255 bytes for the filename.

ZFS

I’ve updated an article explaining how to create a ZPool raidz (RAID 5 equivalent) from three loop devices based on local files.

Thanks to those that bought my ZFS book this month. :)

HTML and JavaScript

I wrote a super simple code to hide the <p> using jQuery and JavaScript.

Free books

https://books.goalkicker.com/

Books I bought

This month I bought these books.

Firewall

I continued to block any Russian or Belarus Ip Address that connects to the blog.

I also started to block entire ranges of Ip’s from Digital Ocean, as many attacks come from Servers in their infrastructure.

Despite blocking tens of thousands of Ip Addresses, the number of visitors keep growing.

My Health

Thanks to my strict discipline I managed to recover super well and I’m healthier than before and guided by the satisfied doctors we removed two daily medicines.

I started a new medicine that is for the final phase of my recuperation, which doctors expect to be completely. In fact I’m much more healthier than I was before going to the hospital.

Humor

Will AI take the world?
Sadly, true history. The responsibility to deliver is from all of us.

The Cloud is for Scaling

Last Update: 2022-08-03 I added the Enterprises, as more and more services have been provided by Amazon and the other major Cloud Providers and they offer a suitable solution for Enterprises requiring high availability in services managed by Amazon, without the need to configure and maintain their own setups manually.

dell-blades-m4110The Cloud is for Startups, and for Scaling. Nothing more.

In the future will be used by phone operators, to re-dimension their infrastructure and bandwidth in real time according to demand, but nowadays the Cloud is for Startups.

Examine the prices in my post in cmips, take a look, examine the performance also of the different CPU. You see that according to CMIPS v.1.03 a Desktop Processor Intel i7-4770S, worth USD $300, performs better than an Amazon M2 High Memory Quadruple Extra Large and than a Rackspace First gen. 30 GB RAM 8 Cores?.

Today the public cost of an Amazon M2 High Memory Quadruple Extra Large running for a month is USD $1,180.80 so USD $1.64 per hour and the Rackspace First Generation 30 GB RAM 8 Cores 1200 GB of disk costs is USD $1,425.60 so USD $1.98 per hour running.

And that’s the key, the cost per hour.

Because the greatness, the majesty of the Cloud is that you pay per hour, you pay as you need, or as you go. No attaching contracts. All on demand.

I had my company at a time where the hosting companies and the Data Centers were forcing customers to sign yearly contracts. What if a company only needs to host their Servers for three months? What if they have to close?. No options. You take it or you leave it.

Even renting a dedicated hosting was for at least a month or more, and what if the latency was not good? What if the bandwidth of the provider was not enough?.

Amazon irrupted in the market with strength. I really like that company because they grew the best eCommerce company for buying books, they did a system that really worked, and was able to recommend very useful computer books, and the delivery, logistics was so good, also post-sales service. They simply started to rent the same infrastructure they were using to attend their millions of customers and was a total success.

And for a while few people knew about Amazon deep technologies and functionalities, but later became a fashion.

Now people is using Amazon or whatever provider/Service that contains the word “Cloud” because the Cloud is in the mouth of everyone. Magazines and newspapers speak about the Cloud, so many many companies use it simply because everyone is talking about the Cloud. And those ISP that didn’t had a Cloud have invested heavily to create a Cloud, just because they didn’t want to be the ones without a Cloud, since everyone was asking for it and all the ISP companies were offering their “Clouds”.

Every company claims to have “Cloud” where the only many of them have is Vmware servers, Xen servers, Open Stack… running the tenants or instances of the customers always on the same host servers. No real Cloud, professional Cloud, abstract layered in a Professional way like Amazon, only the traditional “shared hosting” with another name, sharing CPU and RAM and Disk storage using virtual machines called instances.

So, Cloud fashion has become a confusing craziness where no one knows why they are in the Cloud but they believe they have to be in.

But do companies need the Cloud?. Cloud instances?

It depends. The best would be to ask that companies Why you choose the Cloud?.

If you compare the cost of having an instance in the Cloud, is much much more expensive than having a dedicated server. And for that high cost you don’t get more performance.

Virtualization is always slower and disk speed is always an issue in Cloud providers, where all the data travels via network from the disk cabins NAS to the Host servers running the guest instances. Data cannot be at local disks, since every time you start an instance, the resources like CPU and RAM are provisioned, and your instance run in totally different hardware. Only your data remain in the NAS (Network Attached Storage).

So unless you run your in-the-Cloud instance in a special provider that offers local disks, like DigitalOcean that offers SSD but monthly paying, (and so you pay the price by losing the hardware abstraction capability because you’re attached to the CPU that has the disk connected, and also you loss the flexibility of paying per hour of use, as you go), then you’ll face a bottleneck that is the hard disk performance (that for real takes all the data from NAS, where is stored, through the local network).

So what are the motivations to use the Cloud?. I try to put some examples, out of these it has no much sense, I think. You can send me your happy-in-Cloud scenarios if you found other good uses.

Example A) Saving initial costs, avoid contract attachment and grow easily own-made

Imagine a Developer that start its own project. May be it works, may be not, but instead of having a monthly contract for a dedicated server, he starts with an Amazon Free Tier (better not, use Small instance at least) and runs a web. If it does not work, simply stop the instance and pay no more. If the project works and has more and more users he can re-dimension the server with a click. Just stop the instance, change the type of instance, start it again with more RAM and more CPU power. Fast.

Hiring a dedicated server implies at least monthly contracts, average of USD $100 per month, and is not easy to move to a bigger server, not fast and is expensive as it requires the ISP tech guys to move the data, to migrate from a Server to another.

Also the available bandwidth is to be taken in consideration. Bandwidth is expensive and Amazon can offer 150 Mbit to smaller machines. Not all the Internet Service Providers can offer that bandwidth even with most advanced packets.

If the project still grows, with a click, in seconds, 20 instances with a lot of bandwidth can be deployed and serving traffic to your customers very quick.

You save the init costs of buying Servers, and the time to deal with hardware, bandwidth limitations and avoid contracts, but you pay an hourly rate a lot more expensive. So in the long run is much much expensive using Amazon and less powerful than having dedicated servers. That happened to Zynga, that was paying $63M annually to Amazon and decided to step back from Amazon to their own Data Centers again. (another fortune tech link)

The limited CPU power was also a deal breaker for many companies that needed really powerful CPU and gigs of RAM for their Database Servers. Now this situation is much better with the introduction of the new Servers.

This developer can benefit from doing bacups with a click, cloning, starting instances from an image, having more static ip’s with a click, deploying built-in (from the Cloud provider) load balancers, using monitoring services like CloudWatch, creating Volumes and attaching to the servers for additional space…

Example B) An Startup with fluctuating number of users and hopes of growing

Imagine an Startup with a wonderful Facebook Application.

During 80% of the day has few visits, may be only need 3 Servers, but during 20% of the hours of the day from 10:00 to 15:00 users connect like hell, so they need 20 servers to attend this traffic and workload, and may be tomorrow needs 30 servers.

With the Cloud they pay for 3 servers 24 hours per day and for the other 17 servers only pay the hours they are on, that’s 5 hours per day. Doing that they save money and they have an unlimited * amount of power. (* There are limits for real, you have to specially request authorisation to run more than default max. servers for the zone, that is normally 20 instances for Amazon. Also it can happen theoretically that when you request new instances the Zone has no instances available).

So well, for an Startup growing, avoiding hiring 20 dedicated servers and instead running into the Cloud as many as they need, for just the time they need, Auto-Scaling up and down, and can use the servers NOW and pay the next month with Visa card, all of that can make a difference for a growing Startup.

If the servers chosen are not powerful enough that is solved with a click, changing instance type. So fast. A minute.

It’s only a matter of money.

Example C) e-Learning companies and online universities

e-Learning platforms also get benefits from the Auto-Scaling for the full occupation hours.

The built-in functionalities of the Cloud to clone instances is very useful to deploy new web servers, or new environments for students doing practices, in the case of teaching Information Technology subjects, where the users need to practice against a real server (Linux or windows).

Those servers can be created and destroyed, cloned from the main -ready to go- template. And also servers can be scheduled to stop at a certain hour and to start also, so saving the money from the hours not needed.

Example D) Digital agencies, sports and other events

When there is an Special event, like motorcycle running, when a Football Team scores, when there is an spot in tv announcing a product…

At those moments the traffic to the site can multiply, so more servers and more bandwidth have to be deployed instantly. That cannot be done with physical servers, hardware, but is very easy to provision instances from the Cloud.

Mass mailing email campaigns can also benefit from creating new Servers when needed.

Example E) Proximity and SEO

Cloud providers have Data Centres everywhere. If you want to have servers in Asia, or static content to be deployed faster, or in South-America, or in Europe… the Cloud providers have plenty of Data Centers all over the world.

Example F) Game aficionado and friends sharing contents

People that loves cooperative games can find the needed hungry bandwidth and at a moderate price. If they run their private server few hours, at night, from 22:00 to 01:00 as example, they will benefit from a great bandwidth from the big Cloud provider and pay only 3 hours per day (the exceed of traffic uses to be paid in most providers, but price of additional GB uses to be really really competitive).

Friends sharing contents in an Ftp also, can benefit from this Cloud servers, but probably they will find more easy to use services like Dropbox.

Example G) Startup serving contents

An Startup serving videos, images, or books, can benefit not only from the great bandwidth of big Cloud providers (this has been covered before), but for a very cheap price for exceeding Gigabyte transferred.

Local ISP can’t offer 150 Mbit for an instance of USD $20 and USD $0.12 per additional GB transferred.

Many Cloud providers also allow unlimited incoming traffic from the Internet, and from Server to Server through private ip’s.

Example H) Enterprises that want to outsource the solutions and having better availability

Since I wrote this article in 2013, many CSP have been incorporation more and more powerful services to their catalalog.

For instance Amazon has service like DynamoDB, RDS, Load Balancers able to balance servers in different availability zones, with Auto-Scaling, SQS for Message Queues, ElastiCache, Lambda, S3…

Many Enterprise find convenient to have all their services offloaded to Amazon, Google Cloud, Microsoft Azure… so they require less Operations and System Administration Engineers.

Other cases

For other cases Dedicated Servers are much more Powerful, faster and cheaper, at the price of being “static” in the sense of attached, not layer abstracted, but all the aspects of your Project have to be taken in count before deciding stepping into or out of the Cloud.

In general terms I would say that the Cloud is for Scaling, but in 2022 it’s also to outsource Operation Services, to fully managed services by the CSPs.