Category Archives: Troubleshoot

Things I hate from PHP

I love PHP, is fast to develop, it has many useful built-in features, can be extended with C modules, Arrays are wonderful and saves a lot of time from data-type conversion, but there are certain problems that you should know and that you should be aware of.

There are wonderful posts that mention a lot of issues with PHP, in this article I mention only the stuff I’ve not seen around.

I recommend you reading this wonderful post about the bad design on PHP:

PHP a fractal of bad design

this-is-php-very-bad

‘string’ == 0 is (often) true

This is something everyone has fell some times. And still see a lot of code on GitHub and in my new Teams when I go to a project that fall to that problem.

PHP is “clever” transforming the type of data to compare it. This allow to produce code much faster (try to parse floats in Java or C++ in web projects) but also leads to problems some times.

So if your developers write a code like this, that gets the income sent from a web form:

$s_income = $_POST['income'];
if ($s_income == 0) {
    // The guy is poor, save to the evil CRM database as no interesting person...
    // ...
}

That will fail detecting as 0 if someone enters in the texbox ‘milions’ or ‘$100000’.

The solution would be to check if $s_income is empty, and if is not empty but intval($s_income) is 0, request the user to reinput only with numeric values.

Another example. Imagine that you have a program that reads a CSV file, that has addresses. For example seven lines:

Facebook,Hacker Way,1,Menlo Park,94025,CA
Amazon,2nd Avenue,,1516,WA
Microsoft Corporation,One Microsoft Way,0,,WA
Apple,Infinite Loop, 1,Cupertino,95014,CA
Twitter,Market Street,  1355,,,CA
Netflix,Winchester Circle,one hundred,Los Gatos,95032,CA
Cmips,,0-1,Palo Alto,,CA
Fake Address,Nowhere,Building 5,Silicon Valley,,CA

CSV may contain errors, as most of times the data comes from data input manually at some moment or entered by users via web.

So your code reads it, puts each field in an array field as string, and you can use it.

Let’s assume that our buggy code looks for a 0 in the number, and then performs some action like setting a boolean to FALSE in the database or whatever.

Something like:

<?php
$i_row = 0;
if (($o_handle = fopen("addresses.csv", "r")) !== FALSE) {
    while (($st_data = fgetcsv($o_handle, 1000, ",")) !== false) {
        $i_row++;
        $i_num_fields = count($st_data);

        $s_company_name = $st_data[0];
        $s_street = $st_data[1];
        $s_number = $st_data[2];

        if ($s_number == 0) {
            // The address has no number
            // Do something real...
            echo 'Row: '.$i_row.' '.$s_company_name.' The address has no number! read ('.$s_number.')'."\n";
        } else {
            echo 'Row: '.$i_row.' '.$s_company_name.' number '.$s_number.' found!'."\n";
        }
    }
    fclose($o_handle);
}

And this is the result:

blog-carlesmateo-com-things-i-hate-from-php-sample-equal-0
Look at the results:

Expression Result by PHP
 ‘1’ == 0  FALSE
 ” == 0  TRUE
 ‘ ‘ == 0  (space)  TRUE
 ‘ 1’ == 0  (space and 1)  FALSE
 ‘  1355’ == 0  (space space and 1355)  FALSE
 ‘one hundred’ == 0  TRUE
 ‘0-1’ == 0  TRUE
 ‘Building 5’ == 0  TRUE

If you do ‘one string’ == 0 it returns TRUE, but, the mechanichs of why that happens are curious, capricious and quasi-random.

Normally the mechanichs are: PHP sees that has to evaluate a string to a number and converts the string to a number via intval, so ‘  1355’ == 0 is true because intval(‘  1355’) returns 1355. Please note that ‘  1355’ has two spaces.

Ok. That explains everything, but still is dangerous because ‘Building 5’ == 1 returns FALSE but ‘Building 5’ == 0 returns TRUE, so most Junior developers (and many self-called Seniors) will use that instead of ‘Building 5’ === ” empty string.

This is funny, but is more funny when we introduce another line to the CSV:

Fake Address outside US,Somewhere,.1,Andorra,AD100,Andorra

Here we introduced dot one ‘.1’ and when  run the program it detects as a number, so is not doing intval(‘.1’) but floatval(‘.1’) that returns 0.1

blog-carlesmateo-com-equal-dot-oneI introduced a postal code from Andorra because they start with ‘AD’, so ‘AD100’ in the example.

This is to demonstrate that our program would have detected the Postal Codes from US as numbers, but when used to deal with data from other countries would had failed as ‘AD100’ == 0 TRUE.

So always use === to check the type also and do the intval.

In this sample:

if ( !empty($s_postal_code) && intval($s_postal_code) === 0) {

// Wops! The postal code is there but is not a number

}

Also to check the input data to be sure that match the requirements, would have discovered future weird cases like postal codes with letters. Sample:

if (intval($s_postal_code) != $s_postal_code) {

// Wops! The code is not only numeric

}

Many professional people has explained the crazyness about that magic conversion and ‘string’ == 0, so I will not use more time.

Other crazy results

EXPRESSION RESULT BY PHP
 ‘1’ == ‘ 1’ (space and 1)  TRUE
 ‘1’ == ‘          1) (several spaces and 1)  TRUE
 ‘1’ == ‘+1’  (plus sign and 1)  TRUE
 ‘-1’ == ‘   -1’  (spaces and -1)  TRUE
 ‘-1’ == ‘                                        -1.00’  (spaces -1 dot 00)  TRUE
 ‘-1’ == ‘                                        -1.000000000000001’  FALSE
 ‘-1’ == ‘                                        -1.000000000000000000000000000000000000000000000001’  TRUE
 ‘1.0’ == ‘1.000000000000000’  TRUE
 ‘1e10’ == “1000000000”  TRUE
 ‘1e1’ == “0x0A”  TRUE
 -1 == ‘                                            -1’ (integer -1 equal to string with spaces and -1)  TRUE

So if you register in a PHP system with username 12345 you will be able to login later by entering [space][space][space][space][space]+12345.00000000 or if you pick an username like 1000000000 you’ll be able to login just by entering 1e10 (what could be very bad if there is another user in the system with username 1e10).
So always use the === to check values.

The “amazing” world of ++

Try to add ++ to an string, and to a string that contains decimal symbols…

carlesmateo-com-i-hate-from-php-randomnessity

The ‘horrible’, the floats

As the PHP documentation page says:

Warning

Never cast an unknown fraction to integer, as this can sometimes lead to unexpected results.

carlesmateo-com-from-php-net

So yes…

carlesmateo-com-php-with-floats

 

It is shocking that var_export and var_dump show different values, but more shocking is to get this:

php > $i_valor_float = 81.60;
php > echo intval($i_valor_float * 100).”\n”;

And getting 8159

carlesmateo-com-php-float-losing-cents

If you are and e-commerce or a bank losing cents you’ll not be happy.

In fact, the TPV Visa payment for Sermepa is as buggy as:

//-////////////////////////////////////////////
//desc: Asignamos el importe de la compra
//param: importe: total de la compra a pagar
//return: Retornamos el importe ya modificado
public function importe( $importe = 0 )
{
    $importe = $this->parseFloat( $importe );
    
    // sermepa nos dice Para Euros las dos últimas posiciones se consideran decimales.
    $importe = intval( $importe*100 );

You will have to do some workaround, use strval instead of intval:

php > $i_value_float = 81.60;
php > echo intval($i_value_float * 100).”\n”;
8159
php > echo (9000 – intval($i_value_float * 100)).”\n”;
841
php > echo strval($i_value_float * 100).”\n”;
8160
php > echo (9000 – strval($i_value_float * 100)).”\n”;
840

So in the Sermepa’s code you can do:

    $importe = strval($importe*100);

Or:

    $importe = intval(strval($importe*100));

carlesmateo-com-php-float-intval-lossing-strval-working

Float offers so bizarre scenarios like (code copied from contribution in PHP.net help):
$x = 8 – 6.4; // which is equal to 1.6
$y = 1.6;
var_dump($x == $y);

More funny is to do:
echo $x-$y;

You get:
-4.4408920985006E-16

More information on PHP float here: http://php.net/manual/en/language.types.float.php

Arrays keys being overwritten – part 1

As you know the arrays in PHP can be numeric or string.

As described in PHP documentation about arrays:

The key can either be an integer or a string. The value can be of any type.

Additionally the following key casts will occur:

  • Strings containing valid integers will be cast to the integer type. E.g. the key “8” will actually be stored under 8. On the other hand “08” will not be cast, as it isn’t a valid decimal integer.
  • Floats are also cast to integers, which means that the fractional part will be truncated. E.g. the key 8.7 will actually be stored under 8.
  • Bools are cast to integers, too, i.e. the key true will actually be stored under 1 and the key false under 0.
  • Null will be cast to the empty string, i.e. the key null will actually be stored under “”.
  • Arrays and objects can not be used as keys. Doing so will result in a warning: Illegal offset type.

If multiple elements in the array declaration use the same key, only the last one will be used as all others are overwritten.

But take a look to the following code:

<?php

$st_array = Array('+1' => 'This is key +1',
                  '*1' => 'This is key *1',
                  '-1' => 'This is key -1');

var_dump($st_array);

Look at the dump:

blog-carlesmateo-com-why-i-hate-php-array-keysSo key of the type string ‘-1’ has been converted to integer key -1.

Fut funny, ‘+1’ has been kept as string ‘+1’. If you do:

echo intval(‘+1’);

You get integer 1. So what is the exact function used to check the key to know if it can be casted to integer remains a mistery.

Arrays keys being overwritten – part 2

Taking the CSV sample file from before, and using the Postal Codes one may notice another problem.

When we load the postal code ‘94025’ it is added to the array as integer key, not string key.

This is a problem if we do an array_merge, because:

Values in the input array with numeric keys will be renumbered with incrementing keys starting from zero in the result array.

Creating an array with key string postal code and merging with another array of postal codes will cause the loss of the index. That is very bad.

There are much more problems.

In Barcelona we have many postal codes starting with 0 like ‘08014’, so those will remain as string key. As said in Andorra there are like ‘AD100’.

So if we have a program like that:

<?php

$st_array = Array( '08014' => 'Postal code from Barcelona, Catalonia',
                   '94025' => 'Postal code from Menlo Park, California',
                   'AD100' => 'Postal code from Andorra');

ksort($st_array);

var_dump($st_array);

We will get:

blog-carlesmateo-com--things-i-hate-from-php-array-sortSo we got the string sorted, and later the numeric keys sorted. So bad, as ‘08014’ would be expected to be before ‘94025’, but as the last was casted o integer and the first kept as string key, that caused a bizarre result in our postal code sorting program.

That can be very annoying in many cases, imagine you read a fixed length CSV file or you read from a Webservice, XML or databases the age of people stored with 2 digit so ’10’ means ten years old, ’99’ ninety-nine, etc… you will have the guys from 0 to 9 years old (00 to 09) kept as string, while the others as number. Now do an array_merge with another array and the mess is served.

I recommend adding a character infront of your keys to ensure that cast will not be produced, so use ‘C94024’ and ‘C08014’ and ‘CAD100’… Sort algorithms will work, and you will avoid the problems derivated from merging and sorting keys casted to numeric.

Bizarre behaviour on Boolean execution

Some Sysadmins are used to constructions like this in their bash scripts and Python:

true && do_something(); // This evaluates the first part always and then executes do_something();

file_exists(‘/tmp/whatever’) || touch(‘/tmp/whatever’); // If file_exist returns false so then the second part is executed and the file is created

Although I never found documented this way of executing in PHP, it works, and for sure you have seen samples like that:

$result = mysql_query('SELECT foo FROM bar', $o_db) or die('Query failed: ' . mysql_error($o_db));

 

When working Contractor I found a project using it. It was an habit from the CEO, that came from Systems branch. He had more bad habits like no documenting, programming with vim and refactoring the code of my Team during the weekends with bash replace commands (that caused conflicting variable names, methods, etc…). (* I know is weird that a CEO changes the code but believe me, companies do a lot of crazy things)

He told me that he thought that less lines of code meant more clever developer, and so he was always refactoring clear code to this Boolean a-like execution.

Using that style is bad. It doesn’t allow proper error handling, nor a flux of execution clear nor raising exceptions.

I shown him that this works not like he thought and found samples that crashes the thing:

Trying to throw an exception breaks:

true && throw new Exception('Hello');

If using echo, it breaks:

php-error-unexpectedSo it’s really a bad PHP design.

This behaviour is also funny:

2015-05-05-114624-blog-carlesmateo-com-php-5-5-9

So passing by reference an undefined index, causes it to be created with null without any warning.

I found that in the passing variables documentation, in a contributed comment from 10 years ago!. Still happens with PHP 5.5.

More samples:

blog-carlesmateo-com-passing-array-not-existing-by-ref

 

Troubleshooting apps in Linux

Let’s say you are on a system and a program stops working.

You check the space on disk, check that no one has modified the config files, check things like dns, etc… everything seems normal and you don’t know what else to check.

It could be that the filesystem got corrupted after a powerdown, for example, and one file or more are corrupted and this would be hard to figure out.

To find whats going wrong then you can use strace.

In the simplest case strace runs the specified command until it exits. It intercepts and records the system calls which are called by a process and the signals which are received by a process. The name of each system call, its arguments and its return value are printed on standard error or to the file specified with the -o option.

http://linux.die.net/man/1/strace

As you may know the programs request system calls, and get signals from the Operating System/Kernel.

strace will show all those requests done by the program, and the signals received. That means that you will see the requests from the program to the kernel to open a file, for example a config file.

Executing:

strace /usr/bin/ssh

That is the sample output:

strace /usr/bin/ssh
execve("/usr/bin/ssh", ["/usr/bin/ssh"], [/* 61 vars */]) = 0
brk(0)                                  = 0x7fc71509c000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713cb2000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=126104, ...}) = 0
mmap(NULL, 126104, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc713c93000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240Z\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=134224, ...}) = 0
mmap(NULL, 2234088, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc713870000
mprotect(0x7fc71388f000, 2097152, PROT_NONE) = 0
mmap(0x7fc713a8f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1f000) = 0x7fc713a8f000
mmap(0x7fc713a91000, 1768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc713a91000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\361\5\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1934816, ...}) = 0
mmap(NULL, 4045240, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc713494000
mprotect(0x7fc713646000, 2097152, PROT_NONE) = 0
mmap(0x7fc713846000, 155648, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b2000) = 0x7fc713846000
mmap(0x7fc71386c000, 14776, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc71386c000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14664, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c92000
mmap(NULL, 2109736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc713290000
mprotect(0x7fc713293000, 2093056, PROT_NONE) = 0
mmap(0x7fc713492000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fc713492000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\36\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=100728, ...}) = 0
mmap(NULL, 2195784, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc713077000
mprotect(0x7fc71308f000, 2093056, PROT_NONE) = 0
mmap(0x7fc71328e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fc71328e000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320:\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=97144, ...}) = 0
mmap(NULL, 2202280, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc712e5d000
mprotect(0x7fc712e73000, 2097152, PROT_NONE) = 0
mmap(0x7fc713073000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7fc713073000
mmap(0x7fc713075000, 6824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc713075000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\234\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=252704, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c91000
mmap(NULL, 2348608, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc712c1f000
mprotect(0x7fc712c5a000, 2097152, PROT_NONE) = 0
mmap(0x7fc712e5a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3b000) = 0x7fc712e5a000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\36\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1853400, ...}) = 0
mmap(NULL, 3961912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc712857000
mprotect(0x7fc712a14000, 2097152, PROT_NONE) = 0
mmap(0x7fc712c14000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bd000) = 0x7fc712c14000
mmap(0x7fc712c1a000, 17464, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc712c1a000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\31\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=256224, ...}) = 0
mmap(NULL, 2351392, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc712618000
mprotect(0x7fc712655000, 2097152, PROT_NONE) = 0
mmap(0x7fc712855000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3d000) = 0x7fc712855000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360l\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=135757, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c90000
mmap(NULL, 2212936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc7123fb000
mprotect(0x7fc712412000, 2097152, PROT_NONE) = 0
mmap(0x7fc712612000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fc712612000
mmap(0x7fc712614000, 13384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc712614000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libkrb5.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260p\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=848672, ...}) = 0
mmap(NULL, 2944608, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc71212c000
mprotect(0x7fc7121f1000, 2093056, PROT_NONE) = 0
mmap(0x7fc7123f0000, 45056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc4000) = 0x7fc7123f0000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360;\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=158136, ...}) = 0
mmap(NULL, 2257008, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc711f04000
mprotect(0x7fc711f2a000, 2093056, PROT_NONE) = 0
mmap(0x7fc712129000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x25000) = 0x7fc712129000
mmap(0x7fc71212b000, 112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc71212b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libcom_err.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\24\0\0\0\0\0\0"..., 832) = 832
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c8f000
fstat(3, {st_mode=S_IFREG|0644, st_size=14592, ...}) = 0
mmap(NULL, 2109896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc711d00000
mprotect(0x7fc711d03000, 2093056, PROT_NONE) = 0
mmap(0x7fc711f02000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fc711f02000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@ \0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31160, ...}) = 0
mmap(NULL, 2126632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc711af8000
mprotect(0x7fc711aff000, 2093056, PROT_NONE) = 0
mmap(0x7fc711cfe000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7fc711cfe000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libkeyutils.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\17\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14256, ...}) = 0
mmap(NULL, 2109456, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc7118f4000
mprotect(0x7fc7118f6000, 2097152, PROT_NONE) = 0
mmap(0x7fc711af6000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fc711af6000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c8e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c8d000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713c8b000
arch_prctl(ARCH_SET_FS, 0x7fc713c8b840) = 0
mprotect(0x7fc712c14000, 16384, PROT_READ) = 0
mprotect(0x7fc711af6000, 4096, PROT_READ) = 0
mprotect(0x7fc713492000, 4096, PROT_READ) = 0
mprotect(0x7fc711cfe000, 4096, PROT_READ) = 0
mprotect(0x7fc712612000, 4096, PROT_READ) = 0
mprotect(0x7fc711f02000, 4096, PROT_READ) = 0
mprotect(0x7fc712129000, 4096, PROT_READ) = 0
mprotect(0x7fc713073000, 4096, PROT_READ) = 0
mprotect(0x7fc7123f0000, 40960, PROT_READ) = 0
mprotect(0x7fc712855000, 4096, PROT_READ) = 0
mprotect(0x7fc712e5a000, 4096, PROT_READ) = 0
mprotect(0x7fc71328e000, 4096, PROT_READ) = 0
mprotect(0x7fc713846000, 110592, PROT_READ) = 0
mprotect(0x7fc713a8f000, 4096, PROT_READ) = 0
mprotect(0x7fc713f1f000, 8192, PROT_READ) = 0
mprotect(0x7fc713cb4000, 4096, PROT_READ) = 0
munmap(0x7fc713c93000, 126104)          = 0
set_tid_address(0x7fc713c8bb10)         = 13672
set_robust_list(0x7fc713c8bb20, 24)     = 0
futex(0x7fff5c43f09c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7fc713c8b840) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7fc7124017e0, [], SA_RESTORER|SA_SIGINFO, 0x7fc71240abb0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fc712401860, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fc71240abb0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
statfs("/sys/fs/selinux", 0x7fff5c43f090) = -1 ENOENT (No such file or directory)
statfs("/selinux", 0x7fff5c43f090)      = -1 ENOENT (No such file or directory)
brk(0)                                  = 0x7fc71509c000
brk(0x7fc7150bd000)                     = 0x7fc7150bd000
open("/proc/filesystems", O_RDONLY)     = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713cb1000
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 1024) = 328
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x7fc713cb1000, 4096)            = 0
open("/dev/null", O_RDWR)               = 3
close(3)                                = 0
openat(AT_FDCWD, "/proc/13672/fd", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
getdents(3, /* 6 entries */, 32768)     = 144
getdents(3, /* 0 entries */, 32768)     = 0
close(3)                                = 0
getuid()                                = 1000
geteuid()                               = 1000
setresuid(-1, 1000, -1)                 = 0
socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3)                                = 0
socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3)                                = 0
open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=513, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc713cb1000
read(3, "# /etc/nsswitch.conf\n#\n# Example"..., 4096) = 513
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0x7fc713cb1000, 4096)            = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=126104, ...}) = 0
mmap(NULL, 126104, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc713c93000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_compat.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\23\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=35728, ...}) = 0
mmap(NULL, 2131288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc7116eb000
mprotect(0x7fc7116f3000, 2093056, PROT_NONE) = 0
mmap(0x7fc7118f2000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7fc7118f2000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnsl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`A\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=97296, ...}) = 0
mmap(NULL, 2202360, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc7114d1000
mprotect(0x7fc7114e8000, 2093056, PROT_NONE) = 0
mmap(0x7fc7116e7000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7fc7116e7000
mmap(0x7fc7116e9000, 6904, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc7116e9000
close(3)                                = 0
mprotect(0x7fc7116e7000, 4096, PROT_READ) = 0
mprotect(0x7fc7118f2000, 4096, PROT_READ) = 0
munmap(0x7fc713c93000, 126104)          = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=126104, ...}) = 0
mmap(NULL, 126104, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc713c93000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_nis.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240!\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=47760, ...}) = 0
mmap(NULL, 2143616, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc7112c5000
mprotect(0x7fc7112d0000, 2093056, PROT_NONE) = 0
mmap(0x7fc7114cf000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa000) = 0x7fc7114cf000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\"\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=52160, ...}) = 0
mmap(NULL, 2148504, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc7110b8000
mprotect(0x7fc7110c4000, 2093056, PROT_NONE) = 0
mmap(0x7fc7112c3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xb000) = 0x7fc7112c3000
close(3)                                = 0
mprotect(0x7fc7112c3000, 4096, PROT_READ) = 0
mprotect(0x7fc7114cf000, 4096, PROT_READ) = 0
munmap(0x7fc713c93000, 126104)          = 0
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=1823, ...}) = 0
mmap(NULL, 1823, PROT_READ, MAP_SHARED, 3, 0) = 0x7fc713cb1000
lseek(3, 1823, SEEK_SET)                = 1823
munmap(0x7fc713cb1000, 1823)            = 0
close(3)                                = 0
umask(022)                              = 022
write(2, "usage: ssh [-1246AaCfgKkMNnqsTtV"..., 466usage: ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec]
           [-D [bind_address:]port] [-e escape_char] [-F configfile]
           [-I pkcs11] [-i identity_file]
           [-L [bind_address:]port:host:hostport]
           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]
           [-R [bind_address:]port:host:hostport] [-S ctl_path]
           [-W host:port] [-w local_tun[:remote_tun]]
           [user@]hostname [command]
) = 466
exit_group(255)                         = ?
+++ exited with 255 +++

You can also generate a log with that info:

strace -o test_log.txt /usr/bin/ssh

Let’s pay attention to the open files:

carlesmateo-com-strace-open

Here we can see what files were open, the mode and the result.

So if your program failen opening a certain file you will see it on the traces.

Also we can review the access:

cat test_log.txt | grep access --after-context=2
carlesmateo-com-strace-access

You can specify to trace only certain set of system calls by passing parameter -e trace=open,close,read,write,stat,chmod,unlink or -e trace=network or -e trace=process or -e trace=memory or -e trace=ipc or -e trace=signal etcetera.

Can also dump data read -e read=set or -e write=set for a full hexadecimal and ASCII dump of all the data written to file descriptors listed in the specified  set… or -e signal=set (default signal=all) or even by negation -e signal =! SIGIO (or signal=!io)…

You can also trace libraries with ltrace or processes with ptrace.

But strace is very handy, as you can attach to it and detach and the process will continue working.

To attach to a process use -p and see the flow continuously with -f

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
strace -fp 2418 -s 255
strace -fp 2418 -s 255
strace -fp 2418 -s 255

And see the open files with lsof.

carlesmateo-com-lsof-list-open-files

You can use lsof to see the TCP connections, as socket are also kind of files:

lsof -iTCP:80
carlesmateo-com-lsof-itcp

You can also know information of what process is owner of a tcp/udp connection with ss or the older netstat:

netstat -tnp
carlesmateo-com-netstat-tnp-program-owner-connection

Take a look at ss for advanced sockets inspecting. The most typical call, which is also easy to remember, is:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
ss -plant
ss -plant
ss -plant

Of course you will find very interesting info on /proc pseudo-filesystem.

You can troubleshoot the environment for the process by doing:

strings /proc/1714/environ

Where 1714 is the process id, whatever.

/proc/[pid]/fd/ is a subdirectory containing one entry for each file open by the process, named by its file descriptor, being a symbolic link to the actual file.

/proc/[pid]/fdinfo/ will show information on the flags for the access mode of the open files and /proc/[pid]/io contains input/outputs statistics for the process.

carlesmateo-com-cat-proc-pid-io

My Open Source monitoring project ctop.py gets most of the information from /proc