Category Archives: Software development

Begin developing with Cassandra

This article is contributed to Luna Cloud blog by Carles Mateo.

We architects, developers and start ups are facing new challenges.

We have now to create applications that have to scale and scale at world-wide level.

That puts over the table big and exciting challenges.

To allow that increasing level of scaling, we designed and architect tools and techniques and tricks, but fortunately now there are great products born to scale out and to deal with this problems: like NoSql databases like Cassandra, MongoDb, Riak, Hadoop’s Hbase, Couchbase or CouchDb, NoSql in-Memory like Memcached or Redis, big data solutions like Hadoop, distributed files systems like Hadoop’s HDFS, GlusterFs, Lustre, etc…

In this article I will cover the first steps to develop with Cassandra, under the Developer point of view.

As a first view you may be interested in Cassandra because:

Is a Database with no single point of failure
Where all the Database Servers work in Peer to Peer over Tcp/Ip
Fault-tolerance. You can set replication factor, and the data will be sharded and replicated over different servers and so being resilient to node failures
Because the Cassandra Cluster splits and balances the work across the Cluster automatically
Because you can scale by just adding more nodes to the Cluster, that’s scaling horizontally, and it’s linear. If you double the number of servers, you double the performance
Because you can have cool configurations like multi-datacenter and multi-rack and have the replication done automatically
You can have several small, cheap, commodity servers, with big SATA disks with better result than one very big, very expensive, and unable-to-scale-more server with SSD or SAS expensive disks.
It has the CQL language -Cassandra Query Language-, that is close to SQL
Ability to send querys in async mode (the CPU can do other things while waiting for the query to return the results)

Cassandra is based in key/value philosophy but with columns. It supports multiple columns. That’s cool, as theoretically it supports 2 GB per column (at practical level is not recommended to go with data so big, specially in multi-user environments).

I will not lie to you: It is another paradigm, and comes with a lot of knowledge to acquire, but it is necessary and a price worth to pay for being able of scaling at nowadays required levels.

Cassandra only offers native drivers for: Java, .NET, C++ and Python 2.7. The rest of solutions are contributed, sadly most of them are outdated and unmantained.

You can find all the drivers here:

http://planetcassandra.org/client-drivers-tools/

To develop with PHP

Cassandra has no PHP driver officially, but has some contributed solutions.

By myself I created several solutions: CQLSÍ uses cqlsh to perform queries and interfaces without needing Thrift, and Cassandra Universal Driver is a Web Gateway that I wrote in Python that allows you to query Cassandra from any language, and recently I contributed to a PHP driver that speaks the Cassandra binary protocol (v1) directly using Tcp/Ip sockets.

That’s the best solution for me by now, as it is the fastest and it doesn’t need any third party library nor Thrift neither.

You can git clone it from:

https://github.com/uri2x/php-cassandra

Here we go with some samples:

Create a keyspace

KeySpace is the equivalent to a database in MySQL.

<?php

require_once 'Cassandra/Cassandra.php';

$o_cassandra = new Cassandra();

$s_server_host     = '127.0.0.1';    // Localhost
$i_server_port     = 9042; 
$s_server_username = '';  // We don't use username
$s_server_password = '';  // We don't use password
$s_server_keyspace = '';  // We don't have created it yet

$o_cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);

// Create a Keyspace with Replication factor 1, that's for a single server
$s_cql = "CREATE KEYSPACE cassandra_tests WITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor': 1 };";

$st_results = $o_cassandra->query($s_cql);

We can run it from web or from command line by using:

php -f cassandra_create.php

Create a table

<?php

require_once 'Cassandra/Cassandra.php';

$o_cassandra = new Cassandra();

$s_server_host     = '127.0.0.1';    // Localhost
$i_server_port     = 9042; 
$s_server_username = '';  // We don't use username
$s_server_password = '';  // We don't use password
$s_server_keyspace = 'cassandra_tests';

$o_cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);

$s_cql = "CREATE TABLE carles_test_table (s_thekey text, s_column1 text, s_column2 text,PRIMARY KEY (s_thekey));";

$st_results = $o_cassandra->query($s_cql);

If we don’t plan to insert UTF-8 strings, we can use VARCHAR instead of TEXT type.

Do an insert

In this sample we create an Array of 100 elements, we serialize it, and then we store it.

<?php

require_once 'Cassandra/Cassandra.php';

// Note this code uses the MT notation http://blog.carlesmateo.com/maria-teresa-notation-for-php/
$i_start_time = microtime(true);

$o_cassandra = new Cassandra();

$s_server_host     = '127.0.0.1';    // Localhost
$i_server_port     = 9042; 
$s_server_username = '';  // We don't have username
$s_server_password = '';  // We don't have password
$s_server_keyspace = 'cassandra_tests';  

$o_cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);

$s_time = strval(time()).strval(rand(0,9999));
$s_date_time = date('Y-m-d H:i:s');

// An array to hold a emails
$st_data_emails = Array();

for ($i_bucle=0; $i_bucle<100; $i_bucle++) {
    // Add a new email
    $st_data_emails[] = Array('datetime'  => $s_date_time,
                              'id_email'  => $s_time);

}

// Serialize the Array
$s_data_emails = serialize($st_data_emails);

$s_cql = "INSERT INTO carles_test_table (s_thekey, s_column1, s_column2)
VALUES ('first_sample', '$s_data_emails', 'Some other data');";

$st_results = $o_cassandra->query($s_cql);

$o_cassandra->close();

print_r($st_results);

$i_finish_time = microtime(true);
$i_execution_time = $i_finish_time-$i_start_time;

echo 'Execution time: '.$i_execution_time."\n";
echo "\n";

This insert took Execution time: 0.0091850757598877 seconds executed from CLI (Command line).

If the INSERT works well you’ll have a [result] => ‘success’ in the resulting array.

Do some inserts

Here we do 9000 inserts.

<?php

require_once 'Cassandra/Cassandra.php';

// Note this code uses the MT notation http://blog.carlesmateo.com/maria-teresa-notation-for-php/
$i_start_time = microtime(true);

$o_cassandra = new Cassandra();

$s_server_host     = '127.0.0.1';    // Localhost
$i_server_port     = 9042; 
$s_server_username = '';  // We don't have username
$s_server_password = '';  // We don't have password
$s_server_keyspace = 'cassandra_tests';  

$o_cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);

$s_date_time = date('Y-m-d H:i:s');

for ($i_bucle=0; $i_bucle<9000; $i_bucle++) {
    // Add a sample text, let's use time for example
    $s_time = strval(time());

    $s_cql = "INSERT INTO carles_test_table (s_thekey, s_column1, s_column2)
VALUES ('$i_bucle', '$s_time', 'http://blog.carlesmateo.com');";

    // Launch the query
    $st_results = $o_cassandra->query($s_cql);

}

$o_cassandra->close();

$i_finish_time = microtime(true);
$i_execution_time = $i_finish_time-$i_start_time;

echo 'Execution time: '.$i_execution_time."\n";
echo "\n";

Those 9,000 INSERTs takes 6.49 seconds in my test virtual machine, executed from CLI (Command line).

Do a Select

<?php

require_once 'Cassandra/Cassandra.php';

// Note this code uses the MT notation http://blog.carlesmateo.com/maria-teresa-notation-for-php/
$i_start_time = microtime(true);

$o_cassandra = new Cassandra();

$s_server_host     = '127.0.0.1';    // Localhost
$i_server_port     = 9042; 
$s_server_username = '';  // We don't have username
$s_server_password = '';  // We don't have password
$s_server_keyspace = 'cassandra_tests';  

$o_cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);


$s_cql = "SELECT * FROM carles_test_table LIMIT 10;";

// Launch the query
$st_results = $o_cassandra->query($s_cql);
echo 'Printing 10 rows:'."\n";

print_r($st_results);

$o_cassandra->close();

$i_finish_time = microtime(true);
$i_execution_time = $i_finish_time-$i_start_time;

echo 'Execution time: '.$i_execution_time."\n";
echo "\n";

Printing 10 rows passing the query with LIMIT:

$s_cql = "SELECT * FROM carles_test_table LIMIT 10;";

echoing as array with print_r takes Execution time: 0.01090407371521 seconds (the cost of printing is high).

If you don’t print the rows, it takes only Execution time: 0.00714111328125 seconds.
Selecting 9,000 rows, if you don’t print them, takes Execution time: 0.18086194992065.

Java

The official driver for Java works very well.

The only initial difficulties may be to create the libraries required with Maven and to deal with the different Cassandra native data types.

To make that travel easy, I describe what you have to do to generate the libraries and provide you with a Db Class made by me that will abstract you from dealing with Data types and provide a simple ArrayList with the field names and all the data as String.

Datastax provides the pom.xml for maven so you’ll create you jar files. Then you can copy those jar file to Libraries folder of any project you want to use Cassandra with.

My Db class:

/*
 * By Carles Mateo blog.carlesmateo.com
 * You can use this code freely, or modify it.
 */

package server;

import java.util.ArrayList;
import java.util.List;
import com.datastax.driver.core.*;

/**
 * @author carles_mateo
 */
public class Db {

    public String[] s_cassandra_hosts = null;
    public String s_database = "cchat";
    
    public Cluster o_cluster = null;
    public Session o_session = null;
    
    Db() {
        // The Constructor
        this.s_cassandra_hosts = new String[10];
        
        String s_cassandra_server = "127.0.0.1";
        
        this.s_cassandra_hosts[0] = s_cassandra_server;
        
        this.o_cluster = Cluster.builder()
                                        .addContactPoints(s_cassandra_hosts[0]) // More than 1 separated by comas
                                        .build();
        this.o_session = this.o_cluster.connect(s_database);  // This is the KeySpace

    }
    
    public static String escapeApostrophes(String s_cql) {
        String s_cql_replaced = s_cql.replaceAll("'", "''");
        
        return s_cql_replaced;
    }
    
    public void close() {
        // Destructor calles by the garbagge collector
        this.o_session.close();
        this.o_cluster.close();
    }
    
    public ArrayList query(String s_cql) {
        
        ResultSet rows = null;
        
        rows = this.o_session.execute(s_cql);
        
        ArrayList st_results = new ArrayList();
        List<String> st_column_names = new ArrayList<String>();
        List<String> st_column_types = new ArrayList<String>();

        ColumnDefinitions o_cdef = rows.getColumnDefinitions();

        int i_num_columns = o_cdef.size();
        for (int i_columns = 0; i_columns < i_num_columns; i_columns++) {
            st_column_names.add(o_cdef.getName(i_columns));
            st_column_types.add(o_cdef.getType(i_columns).toString());                
        }                
        
        st_results.add(st_column_names);
        
        for (Row o_row : rows) {
            
            List<String> st_data = new ArrayList<String>();
            for (int i_column=0; i_column<i_num_columns; i_column++) {
                if (st_column_types.get(i_column).equals("varchar") || st_column_types.get(i_column).equals("text")) {
                    st_data.add(o_row.getString(i_column));
                } else if (st_column_types.get(i_column).equals("timeuuid")) {
                    st_data.add(o_row.getUUID(i_column).toString());
                } else if (st_column_types.get(i_column).equals("integer")) {
                    st_data.add(String.valueOf(o_row.getInt(i_column)));
                }
                // TODO: Implement other data types
                
            }
            st_results.add(st_data);
           
        }
        
        return st_results;
        
    }
    
    public static String getFieldFromRow(ArrayList st_results, int i_row, String s_fieldname) {
        
        List<String> st_column_names = (List)st_results.get(0);
        
        boolean b_column_found = false;
        
        int i_column_pos = 0;
        
        for (String s_column_name : st_column_names) {
            if (s_column_name.equals(s_fieldname)) {
                b_column_found = true;
                break;
            }
            i_column_pos++;
        }
        
        if (b_column_found == false) {
            return null;
        }
        
        int i_num_columns = st_results.size();
        
        List<String> st_data = (List)st_results.get(i_row);
        
        String s_data = st_data.get(i_column_pos);
        
        return s_data;
    }
    
}

Python 2.7

There is no currently driver for Python 3. I requested Datastax and they told me that they are working in a new driver for Python 3.

To work with Datastax’s Python 2.7 driver:

1) Download the driver from http://planetcassandra.org/client-drivers-tools/ or git clone from https://github.com/datastax/python-driver

2) Install the dependencies for the Datastax’s driver

Install python-pip (Installer)

sudo apt-get install python-pip

Install python development tools

sudo apt-get install python-dev

This is required for some of the libraries used by original Cassandra driver.

Install Cassandra driver required libraries

sudo pip install futures
sudo pip install blist
sudo pip install metrics
sudo pip install scales

Query Cassandra from Python

The problem is the same as with Java, the different data types are hard to deal with.
So I created a function convert_to_string that converts known data types to String, and so later we will only deal with Strings.

In this sample, the results of the query are rendered in xml or in html.

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
# Use with Python 2.7+

__author__ = 'Carles Mateo'
__blog__ = 'http://blog.carlesmateo.com'

import sys

from cassandra import ConsistencyLevel
from cassandra.cluster import Cluster
from cassandra.query import SimpleStatement

s_row_separator = u"||*||"
s_end_of_row = u"//*//"
s_data = u""

b_error = 0
i_error_code = 0
s_html_output = u""
b_use_keyspace = 1 # By default use keyspace
b_use_user_and_password = 1 # Not implemented yet

def return_success(i_counter, s_data, s_format = 'html'):
    i_error_code = 0
    s_error_description = 'Data returned Ok'

    return_response(i_error_code, s_error_description, i_counter, s_data, s_format)
    return

def return_error(i_error_code, s_error_description, s_format = 'html'):
    i_counter = 0
    s_data = ''

    return_response(i_error_code, s_error_description, i_counter, s_data, s_format)
    return

def return_response(i_error_code, s_error_description, i_counter, s_data, s_format = 'html'):

    if s_format == 'xml':
        print ("Content-Type: text/xml")
        print ("")
        s_html_output = u"<?xml version='1.0' encoding='utf-8' standalone='yes'?>"
        s_html_output = s_html_output + '<response>' \
                                        '<status>' \
                                        '<error_code>' + str(i_error_code) + '</error_code>' \
                                        '<error_description>' + '<![CDATA[' + s_error_description + ']]>' + '</error_description>' \
                                        '<rows_returned>' + str(i_counter) + '</rows_returned>' \
                                        '</status>' \
                                        '<data>' + s_data + '</data>' \
                                        '</response>'
    else:
        print("Content-Type: text/html; charset=utf-8")
        print("")
        s_html_output = str(i_error_code)
        s_html_output = s_html_output + '\n' + s_error_description + '\n'
        s_html_output = s_html_output + str(i_counter) + '\n'
        s_html_output = s_html_output + s_data + '\n'

    print(s_html_output.encode('utf-8'))
    sys.exit()
    return

def convert_to_string(s_input):
    # Convert other data types to string

    s_output = s_input

    try:
        if value is not None:

            if isinstance(s_input, unicode):
                # string unicode, do nothing
                return s_output

            if isinstance(s_input, (int, float, bool, set, list, tuple, dict)):
                # Convert to string
                s_output = str(s_input)
                return s_output

            # This is another type, try to convert
            s_output = str(input)
            return s_output

        else:
            # is none
            s_output = ""
            return s_output

    except Exception as e:
        # Were unable to convert to str, will return as empty string
        s_output = ""

    return s_output

def convert_to_utf8(s_input):
    return s_input.encode('utf-8')

# ********************
# Start of the program
# ********************

s_format = 'xml'  # how you want this sample program to output

s_cql = 'SELECT * FROM test_table;'
s_cluster = '127.0.0.1'
s_port = "9042" # default port
i_port = int(s_port)

b_use_keyspace = 1
s_keyspace = 'cassandra_tests'
if s_keyspace == '':
    b_use_keyspace = 0

s_user = ''
s_password = ''
if s_user == '' or s_password == '':
    b_use_user_and_password = 0

try:
    cluster = Cluster([s_cluster], i_port)
    session = cluster.connect()
except Exception as e:
    return_error(200, 'Cannot connect to cluster ' + s_cluster + ' on port ' + s_port + '.' + e.message, s_format)

if (b_use_keyspace == 1):
    try:
        session.set_keyspace(s_keyspace)
    except:
        return_error(210, 'Keyspace ' + s_keyspace + ' does not exist', s_format)

try:
    o_results = session.execute_async(s_cql)
except Exception as e:
    return_error(300, 'Error executing query. ' + e.message, s_format)

try:
    rows = o_results.result()
except Exception as e:
    return_error(310, 'Query returned result error. ' + e.message, s_format)

# Query returned values
i_counter = 0
try:
    if rows is not None:
        for row in rows:
            i_counter = i_counter + 1

            if i_counter == 1 and s_format == 'html':
                # first row is row titles
                for key, value in vars(row).iteritems():
                    s_data = s_data + key + s_row_separator

                s_data = s_data + s_end_of_row

            if s_format == 'xml':
                s_data = s_data + ''

            for key, value in vars(row).iteritems():
                # Convert to string numbers or other types
                s_value = convert_to_string(value)
                if s_format == 'xml':
                    s_data = s_data + '<' + key + '>' + '<![CDATA[' + s_value + ']]>' + ''
                else:
                    s_data = s_data + s_value
                    s_data = s_data + s_row_separator


            if s_format == 'xml':
                s_data = s_data + ''
            else:
                s_data = s_data + s_end_of_row

except Exception as e:
    # No iterable data
    return_success(i_counter, s_data, s_format)

# Just print the data
return_success(i_counter, s_data, s_format)

If you did not create the namespace like in the samples before, change those lines to:

s_cql = 'CREATE KEYSPACE cassandra_tests WITH REPLICATION = { \'class\': \'SimpleStrategy\', \'replication_factor\': 1 };'
s_cluster = '127.0.0.1'
s_port = "9042" # default port
i_port = int(s_port)

b_use_keyspace = 1
s_keyspace = ''

Run the program to create the Keyspace and you’ll get:

carles@ninja8:~/Desktop/codi/python/test$ ./lunacloud-create.py Content-Type: text/xml <error_code>0<error_description>

Then you can create the table simply by setting:

s_cql = 'CREATE TABLE test_table (s_thekey text, s_column1 text, s_column2 text,PRIMARY KEY (s_thekey));'
s_cluster = '127.0.0.1'
s_port = "9042" # default port
i_port = int(s_port)

b_use_keyspace = 1
s_keyspace = 'cassandra_tests'

Cassandra Universal Driver

As mentioned above if you use a language Tcp/Ip enabled very new, or very old like ASP or ColdFusion, or from Unix command line and you want to use it with Cassandra, you can use my solution http://www.cassandradriver.com/.

It is basically a Web Gateway able to speak XML, JSon or CSV alike. It relies on the official Datastax’s python driver.

It is not so fast as a native driver, but it works pretty well and allows you to split your architecture in interesting ways, like intermediate layers to restrict even more security (For example WebServers may query the gateway, that will enstrict tome permissions instead of having direct access to the Cassandra Cluster. That can also be used to perform real-time map-reduce operations on the amount of data returned by the Cassandras, so freeing the webservers from that task and saving CPU).

Tip: If you use Cassandra for Development only, you can limit the amount of memory used by editing the file /etc/cassandra/cassandra-env.sh and hardcoding:

    # limit the memory for development environment
    # --------------------------------------------
    system_memory_in_mb="512"
    system_cpu_cores="1"
    # --------------------------------------------

Just before the line:

# set max heap size based on the following

That way Cassandra will believe your system memory is 512 MB and reserve only 256 MB for its use.

Things I hate from PHP

I love PHP, is fast to develop, it has many useful built-in features, can be extended with C modules, Arrays are wonderful and saves a lot of time from data-type conversion, but there are certain problems that you should know and that you should be aware of.

There are wonderful posts that mention a lot of issues with PHP, in this article I mention only the stuff I’ve not seen around.

I recommend you reading this wonderful post about the bad design on PHP:

PHP a fractal of bad design

‘string’ == 0 is (often) true

This is something everyone has fell some times. And still see a lot of code on GitHub and in my new Teams when I go to a project that fall to that problem.

PHP is “clever” transforming the type of data to compare it. This allow to produce code much faster (try to parse floats in Java or C++ in web projects) but also leads to problems some times.

So if your developers write a code like this, that gets the income sent from a web form:

$s_income = $_POST['income'];
if ($s_income == 0) {
    // The guy is poor, save to the evil CRM database as no interesting person...
    // ...
}

That will fail detecting as 0 if someone enters in the texbox ‘milions’ or ‘$100000’.

The solution would be to check if $s_income is empty, and if is not empty but intval($s_income) is 0, request the user to reinput only with numeric values.

Another example. Imagine that you have a program that reads a CSV file, that has addresses. For example seven lines:

Facebook,Hacker Way,1,Menlo Park,94025,CA
Amazon,2nd Avenue,,1516,WA
Microsoft Corporation,One Microsoft Way,0,,WA
Apple,Infinite Loop, 1,Cupertino,95014,CA
Twitter,Market Street,  1355,,,CA
Netflix,Winchester Circle,one hundred,Los Gatos,95032,CA
Cmips,,0-1,Palo Alto,,CA
Fake Address,Nowhere,Building 5,Silicon Valley,,CA

CSV may contain errors, as most of times the data comes from data input manually at some moment or entered by users via web.

So your code reads it, puts each field in an array field as string, and you can use it.

Let’s assume that our buggy code looks for a 0 in the number, and then performs some action like setting a boolean to FALSE in the database or whatever.

Something like:

<?php
$i_row = 0;
if (($o_handle = fopen("addresses.csv", "r")) !== FALSE) {
    while (($st_data = fgetcsv($o_handle, 1000, ",")) !== false) {
        $i_row++;
        $i_num_fields = count($st_data);

        $s_company_name = $st_data[0];
        $s_street = $st_data[1];
        $s_number = $st_data[2];

        if ($s_number == 0) {
            // The address has no number
            // Do something real...
            echo 'Row: '.$i_row.' '.$s_company_name.' The address has no number! read ('.$s_number.')'."\n";
        } else {
            echo 'Row: '.$i_row.' '.$s_company_name.' number '.$s_number.' found!'."\n";
        }
    }
    fclose($o_handle);
}

And this is the result:

Look at the results:

Expression	Result by PHP
‘1’ == 0	FALSE
” == 0	TRUE
‘ ‘ == 0 (space)	TRUE
‘ 1’ == 0 (space and 1)	FALSE
‘ 1355’ == 0 (space space and 1355)	FALSE
‘one hundred’ == 0	TRUE
‘0-1’ == 0	TRUE
‘Building 5’ == 0	TRUE

If you do ‘one string’ == 0 it returns TRUE, but, the mechanichs of why that happens are curious, capricious and quasi-random.

Normally the mechanichs are: PHP sees that has to evaluate a string to a number and converts the string to a number via intval, so ‘ 1355’ == 0 is true because intval(‘ 1355’) returns 1355. Please note that ‘ 1355’ has two spaces.

Ok. That explains everything, but still is dangerous because ‘Building 5’ == 1 returns FALSE but ‘Building 5’ == 0 returns TRUE, so most Junior developers (and many self-called Seniors) will use that instead of ‘Building 5’ === ” empty string.

This is funny, but is more funny when we introduce another line to the CSV:

Fake Address outside US,Somewhere,.1,Andorra,AD100,Andorra

Here we introduced dot one ‘.1’ and when run the program it detects as a number, so is not doing intval(‘.1’) but floatval(‘.1’) that returns 0.1

I introduced a postal code from Andorra because they start with ‘AD’, so ‘AD100’ in the example.

This is to demonstrate that our program would have detected the Postal Codes from US as numbers, but when used to deal with data from other countries would had failed as ‘AD100’ == 0 TRUE.

So always use === to check the type also and do the intval.

In this sample:

if ( !empty($s_postal_code) && intval($s_postal_code) === 0) {

// Wops! The postal code is there but is not a number

}

Also to check the input data to be sure that match the requirements, would have discovered future weird cases like postal codes with letters. Sample:

if (intval($s_postal_code) != $s_postal_code) {

// Wops! The code is not only numeric

}

Many professional people has explained the crazyness about that magic conversion and ‘string’ == 0, so I will not use more time.

Other crazy results

EXPRESSION	RESULT BY PHP
‘1’ == ‘ 1’ (space and 1)	TRUE
‘1’ == ‘ 1) (several spaces and 1)	TRUE
‘1’ == ‘+1’ (plus sign and 1)	TRUE
‘-1’ == ‘ -1’ (spaces and -1)	TRUE
‘-1’ == ‘ -1.00’ (spaces -1 dot 00)	TRUE
‘-1’ == ‘ -1.000000000000001’	FALSE
‘-1’ == ‘ -1.000000000000000000000000000000000000000000000001’	TRUE
‘1.0’ == ‘1.000000000000000’	TRUE
‘1e10’ == “1000000000”	TRUE
‘1e1’ == “0x0A”	TRUE
-1 == ‘ -1’ (integer -1 equal to string with spaces and -1)	TRUE

So if you register in a PHP system with username 12345 you will be able to login later by entering [space][space][space][space][space]+12345.00000000 or if you pick an username like 1000000000 you’ll be able to login just by entering 1e10 (what could be very bad if there is another user in the system with username 1e10).
So always use the === to check values.

The “amazing” world of ++

Try to add ++ to an string, and to a string that contains decimal symbols…

The ‘horrible’, the floats

As the PHP documentation page says:

Warning

Never cast an unknown fraction to integer, as this can sometimes lead to unexpected results.

So yes…

It is shocking that var_export and var_dump show different values, but more shocking is to get this:

php > $i_valor_float = 81.60;
php > echo intval($i_valor_float * 100).”\n”;

And getting 8159

If you are and e-commerce or a bank losing cents you’ll not be happy.

In fact, the TPV Visa payment for Sermepa is as buggy as:

//-////////////////////////////////////////////
//desc: Asignamos el importe de la compra
//param: importe: total de la compra a pagar
//return: Retornamos el importe ya modificado
public function importe( $importe = 0 )
{
    $importe = $this->parseFloat( $importe );
    
    // sermepa nos dice Para Euros las dos últimas posiciones se consideran decimales.
    $importe = intval( $importe*100 );

You will have to do some workaround, use strval instead of intval:

php > $i_value_float = 81.60;
php > echo intval($i_value_float * 100).”\n”;
8159
php > echo (9000 – intval($i_value_float * 100)).”\n”;
841
php > echo strval($i_value_float * 100).”\n”;
8160
php > echo (9000 – strval($i_value_float * 100)).”\n”;
840

So in the Sermepa’s code you can do:

    $importe = strval($importe*100);

Or:

    $importe = intval(strval($importe*100));

Float offers so bizarre scenarios like (code copied from contribution in PHP.net help):
$x = 8 – 6.4; // which is equal to 1.6
$y = 1.6;
var_dump($x == $y);

More funny is to do:
echo $x-$y;

You get:
-4.4408920985006E-16

More information on PHP float here: http://php.net/manual/en/language.types.float.php

Arrays keys being overwritten – part 1

As you know the arrays in PHP can be numeric or string.

As described in PHP documentation about arrays:

The key can either be an integer or a string. The value can be of any type.

Additionally the following key casts will occur:

Strings containing valid integers will be cast to the integer type. E.g. the key “8” will actually be stored under 8. On the other hand “08” will not be cast, as it isn’t a valid decimal integer.

Floats are also cast to integers, which means that the fractional part will be truncated. E.g. the key 8.7 will actually be stored under 8.

Bools are cast to integers, too, i.e. the key true will actually be stored under 1 and the key false under 0.

Null will be cast to the empty string, i.e. the key null will actually be stored under “”.

Arrays and objects can not be used as keys. Doing so will result in a warning: Illegal offset type.

If multiple elements in the array declaration use the same key, only the last one will be used as all others are overwritten.

But take a look to the following code:

<?php

$st_array = Array('+1' => 'This is key +1',
                  '*1' => 'This is key *1',
                  '-1' => 'This is key -1');

var_dump($st_array);

Look at the dump:

So key of the type string ‘-1’ has been converted to integer key -1.

Fut funny, ‘+1’ has been kept as string ‘+1’. If you do:

echo intval(‘+1’);

You get integer 1. So what is the exact function used to check the key to know if it can be casted to integer remains a mistery.

Arrays keys being overwritten – part 2

Taking the CSV sample file from before, and using the Postal Codes one may notice another problem.

When we load the postal code ‘94025’ it is added to the array as integer key, not string key.

This is a problem if we do an array_merge, because:

Values in the input array with numeric keys will be renumbered with incrementing keys starting from zero in the result array.

Creating an array with key string postal code and merging with another array of postal codes will cause the loss of the index. That is very bad.

There are much more problems.

In Barcelona we have many postal codes starting with 0 like ‘08014’, so those will remain as string key. As said in Andorra there are like ‘AD100’.

So if we have a program like that:

<?php

$st_array = Array( '08014' => 'Postal code from Barcelona, Catalonia',
                   '94025' => 'Postal code from Menlo Park, California',
                   'AD100' => 'Postal code from Andorra');

ksort($st_array);

var_dump($st_array);

We will get:

So we got the string sorted, and later the numeric keys sorted. So bad, as ‘08014’ would be expected to be before ‘94025’, but as the last was casted o integer and the first kept as string key, that caused a bizarre result in our postal code sorting program.

That can be very annoying in many cases, imagine you read a fixed length CSV file or you read from a Webservice, XML or databases the age of people stored with 2 digit so ’10’ means ten years old, ’99’ ninety-nine, etc… you will have the guys from 0 to 9 years old (00 to 09) kept as string, while the others as number. Now do an array_merge with another array and the mess is served.

I recommend adding a character infront of your keys to ensure that cast will not be produced, so use ‘C94024’ and ‘C08014’ and ‘CAD100’… Sort algorithms will work, and you will avoid the problems derivated from merging and sorting keys casted to numeric.

Bizarre behaviour on Boolean execution

Some Sysadmins are used to constructions like this in their bash scripts and Python:

true && do_something(); // This evaluates the first part always and then executes do_something();

file_exists(‘/tmp/whatever’) || touch(‘/tmp/whatever’); // If file_exist returns false so then the second part is executed and the file is created

Although I never found documented this way of executing in PHP, it works, and for sure you have seen samples like that:

$result = mysql_query('SELECT foo FROM bar', $o_db) or die('Query failed: ' . mysql_error($o_db));

When working Contractor I found a project using it. It was an habit from the CEO, that came from Systems branch. He had more bad habits like no documenting, programming with vim and refactoring the code of my Team during the weekends with bash replace commands (that caused conflicting variable names, methods, etc…). (* I know is weird that a CEO changes the code but believe me, companies do a lot of crazy things)

He told me that he thought that less lines of code meant more clever developer, and so he was always refactoring clear code to this Boolean a-like execution.

Using that style is bad. It doesn’t allow proper error handling, nor a flux of execution clear nor raising exceptions.

I shown him that this works not like he thought and found samples that crashes the thing:

Trying to throw an exception breaks:

true && throw new Exception('Hello');

If using echo, it breaks:

So it’s really a bad PHP design.

This behaviour is also funny:

So passing by reference an undefined index, causes it to be created with null without any warning.

I found that in the passing variables documentation, in a contributed comment from 10 years ago!. Still happens with PHP 5.5.

More samples: