PHP & Docker

Docker and logs

2017-12-28T13:36:16.901000Z

~~Edit~~

Docker and logs

How docker logging works ?

It’s very simple, to log from a container, you have to write in stdout !

Where container logs are sent ? How can I see it ?

It’s a best practice to log on stdout from a container, it’s an unified interface and directly available from docker logs -f [container] or through docker-compose logs -f. Howerver docker save the output produced by containers in /var/lib/docker/containers/[CONTAINER ID]/[CONTAINER_ID]-json.log by default. Json file is the default log driver, the most easier.

Docker many built in log driver like as following :

none: No logs will be available for the container and docker logs will not return any output.
json-file: The logs are formatted as JSON. The default logging driver for Docker.
syslog: Writes logging messages to the syslog facility. The syslog daemon must be running on the host machine.
journald: Writes log messages to journald. The journald daemon must be running on the host machine.
gelf: Writes log messages to a Graylog Extended Log Format (GELF) endpoint such as Graylog or Logstash.
fluentd: Writes log messages to fluentd (forward input). The fluentd daemon must be running on the host machine.
awslogs: Writes log messages to Amazon CloudWatch Logs.
splunk: Writes log messages to splunk using the HTTP Event Collector.
etwlogs: Writes log messages as Event Tracing for Windows (ETW) events. Only available on Windows platforms.
gcplogs: Writes log messages to Google Cloud Platform (GCP) Logging.
nats: NATS logging driver for Docker. Publishes log entries to a NATS server.

docker logs command is only available with json-file and journald, than mean you need to bring your infrastructure to visualize your logs!

With json-file driver you will out of disk space, mostly if you work with worker (or application) which produce a huge amount of logs. You specify some parameters in order to avoid that.

max-size The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (k, m, or g).

--log-opt max-size=0m

max-file The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. Only effective when max-size is also set. A positive integer.

--log-opt max-file=3

labels Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced log tag options.

--log-opt labels=production_status,geo

env Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced log tag options.

--log-opt env=os,customer

If you want to learn more about docker logging driver and configuration, you can checkout this page https://docs.docker.com/engine/admin/logging/overview/

Logrotate

If you prefer that your host manage logs via logrotate rather than via container definition.

vim /etc/logrotate.d/docker

/var/lib/docker/containers/*/*.log {
  rotate 7
  daily
  compress
  size=1M
  missingok
  delaycompress
  copytruncate
}

And then

logrotate -fv /etc/logrotate.d/docker

PHP, Unicode and UTF-8 without hassles

2017-12-28T13:33:04.899000Z

~~Edit~~

PHP, Unicode and UTF-8 without hassles

Enable mbstring in PHP

PHP doesn’t natively support unicode but it’s compatible, you can check if multibytes functions are available.

php -m | grep mbstring

You can enable it with sudo apt-get install php7.0-mbstring or at compile time by adding this flag --enable-mbstring. More info here. If you can’t install mbstring via those ways, you can use the polyfill provided by symfony.

composer require symfony/polyfill-mbstring

PHP multibyte functions are documented here mb_*

UTF-8 and PHP

php -r "echo ini_get('default_charset');"

If you use PHP EXIF extension you can also check php -r "echo ini_get('exif.encode_unicode');"

Make you application UTF-8 ready !

HTML

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

HTTP

Content-Type: text/html; charset=utf-8

PHP PDO

$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass);
$dbh->exec("SET CHARACTER SET utf8");

$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass, array(
    PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8")
);

Enforce UTF-8

composer require patchwork/utf8

\Patchwork\Utf8\Bootup::initAll();
\Patchwork\Utf8\Bootup::filterRequestUri();
\Patchwork\Utf8\Bootup::filterRequestInputs();
mb_internal_encoding('UTF-8');
mb_http_output('UTF-8');

Never use utf8_decode() and utf8_encode() in your application.

They are misspelled, they only encode and decode between ISO-8859-1 and UTF-8 think about iso88591_to_utf8 for utf8_decode and utf8_to_iso88591 for utf8_encode .

If you want convert to another charset (it’s called transliteration) you can transliterate with iconv

echo iconv('UTF-8', 'ASCII//TRANSLIT', 'text');

$string = '€';
echo iconv(mb_detect_encoding($string), 'ASCII//TRANSLIT', $string);

PHP7, Stay up to date !

2017-12-28T13:45:13.993000Z

~~Edit~~

PHP7, Stay up to date !

TL;DR Full changelog http://php.net/ChangeLog-7.php

New features

[7.1] HTTP/2 server push support in ext/curl

Support for server push has been added to the CURL extension (requires version 7.46 and above). This can be leveraged through the curl_multi_setopt() function with the new CURLMOPT_PUSHFUNCTION constant. The constants CURL_PUSH_OK and CURL_PUSH_DENY have also been added so that the execution of the server push callback can either be approved or denied.

$mh = curl_multi_init();

curl_multi_setopt($mh, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
curl_multi_setopt($mh, CURLMOPT_PUSHFUNCTION, $callback);

$callback = function($parent_ch, $pushed_ch, array $headers) {
    return CURL_PUSH_OK;
};

[RFC] https://wiki.php.net/rfc/curl_http2_push

[7.1] Asynchronous signal handling

A new function called pcntl_async_signals() has been introduced to enable asynchronous signal handling without using ticks (which introduce a lot of overhead).

pcntl_async_signals(true); // turn on async signals

pcntl_signal(SIGHUP,  function($sig) {
    echo "SIGHUP\n";
});

posix_kill(posix_getpid(), SIGHUP);

https://github.com/Seldaek/signal-handler

[7.1] Convert callables to Closures with Closure::fromCallable()

class Test
{
    public function exposeFunction()
    {
        return Closure::fromCallable([$this, 'privateFunction']);
    }

    private function privateFunction($param)
    {
        var_dump($param);
    }
}

$privFunc = (new Test)->exposeFunction();
$privFunc('some value');

[7.1] Support for negative string offsets

Support for negative string offsets has been added to the string manipulation functions accepting offsets, as well as to string indexing with [] or {}. In such cases, a negative offset is interpreted as being an offset from the end of the string.

var_dump("abcdef"[-2]);
var_dump(strpos("aabbcc", "b", -3));

$string = 'bar';
echo "The last character of '$string' is '$string[-1]'.\n";

string (1) "e"
int(3)
The last character of 'bar' is 'r'.

[7.1] Support for keys in list()

$data = [
    ["id" => 1, "name" => 'Tom'],
    ["id" => 2, "name" => 'Fred'],
];

// list() style
list("id" => $id1, "name" => $name1) = $data[0];

// [] style
["id" => $id1, "name" => $name1] = $data[0];

// list() style
foreach ($data as list("id" => $id, "name" => $name)) {
    // logic here with $id and $name
}

// [] style
foreach ($data as ["id" => $id, "name" => $name]) {
    // logic here with $id and $name
}

[7.1] Multi catch exception handling

try {
    // some code
} catch (FirstException | SecondException $e) {
    // handle first and second exceptions
}

[7.1] Iterable pseudo-type

function iterator(iterable $iter)
{
    foreach ($iter as $val) {
        //
    }
}

[7.1] Class constant visibility

class ConstDemo
{
    const PUBLIC_CONST_A = 1;
    public const PUBLIC_CONST_B = 2;
    protected const PROTECTED_CONST = 3;
    private const PRIVATE_CONST = 4;
}

[7.1] Void functions

function swap(&$left, &$right) : void
{
    if ($left === $right) {
        return;
    }

    $tmp = $left;
    $left = $right;
    $right = $tmp;
}

$a = 1;
$b = 2;
var_dump(swap($a, $b), $a, $b);

[7.1] Symmetric array destructuring

$data = [
    [1, 'Tom'],
    [2, 'Fred'],
];

// list() style
list($id1, $name1) = $data[0];

// [] style
[$id1, $name1] = $data[0];

// list() style
foreach ($data as list($id, $name)) {
    // logic here with $id and $name
}

// [] style
foreach ($data as [$id, $name]) {
    // logic here with $id and $name
}

null
int(2)
int(1)

[7.1] Nullable types


function testReturn(): ?string
{
    return 'elePHPant';
}

var_dump(testReturn());

function testReturn(): ?string
{
    return null;
}

var_dump(testReturn());

function test(?string $name)
{
    var_dump($name);
}

test('elePHPant');
test(null);
test();

string(10) "elePHPant"
NULL
string(10) "elePHPant"
NULL
Uncaught Error: Too few arguments to function test(), 0 passed in...

[7.0] Scalar type declarations


declare(strict_type=1);

function test(int $integer, bool $bool, floar $float, string $string)
{

}

[7.0] Return type declarations



function arraysSum(array ...$arrays): array
{
    return array_map(function(array $array): int {
        return array_sum($array);
    }, $arrays);
}

[7.0] Null coalescing operator

$username = $_GET['user'] ?? $_POST['user'] ?? 'nobody';

[7.0] Spaceship operator


// Integers
echo 1 <=> 1; // 0
echo 1 <=> 2; // -1
echo 2 <=> 1; // 1

// Floats
echo 1.5 <=> 1.5; // 0
echo 1.5 <=> 2.5; // -1
echo 2.5 <=> 1.5; // 1

// Strings
echo "a" <=> "a"; // 0
echo "a" <=> "b"; // -1
echo "b" <=> "a"; // 1
?>

[7.0] Constant arrays using define()


define('ANIMALS', [
    'dog',
    'cat',
    'bird'
]);

echo ANIMALS[1]; // outputs "cat"
?>

[7.0] Anonymous classes



$app = new Application;
$app->setLogger(new class implements Logger {
    public function log(string $msg) {
        echo $msg;
    }
});

var_dump($app->getLogger());
?>

[7.0] Unicode codepoint escape syntax

This takes a Unicode codepoint in hexadecimal form, and outputs that codepoint in UTF-8 to a double-quoted string or a heredoc. Any valid codepoint is accepted, with leading 0’s being optional.

echo "\u{aa}";
echo "\u{0000aa}";
echo "\u{9999}";

ª
ª (same as before but with optional leading 0's)
香

[7.0] Closure::call()

Closure::call() is a more performant, shorthand way of temporarily binding an object scope to a closure and invoking it.


class A {private $x = 1;}

// Pre PHP 7 code
$getXCB = function() {return $this->x;};
$getX = $getXCB->bindTo(new A, 'A'); // intermediate closure
echo $getX();

// PHP 7+ code
$getX = function() {return $this->x;};
echo $getX->call(new A);

[7.0] Filtered unserialize()

This feature seeks to provide better security when unserializing objects on untrusted data. It prevents possible code injections by enabling the developer to whitelist classes that can be unserialized.

// converts all objects into __PHP_Incomplete_Class object
$data = unserialize($foo, ["allowed_classes" => false]);

// converts all objects into __PHP_Incomplete_Class object except those of MyClass and MyClass2
$data = unserialize($foo, ["allowed_classes" => ["MyClass", "MyClass2"]]);

// default behaviour (same as omitting the second argument) that accepts all classes
$data = unserialize($foo, ["allowed_classes" => true]);

[7.0] Group use declarations

use some\namespace\{ClassA, ClassB, ClassC as C};
use function some\namespace\{fn_a, fn_b, fn_c};
use const some\namespace\{ConstA, ConstB, ConstC};

[7.0] list() can always unpack objects implementing ArrayAccess

Previously, list() was not guaranteed to operate correctly with objects implementing ArrayAccess. This has been fixed.

[7.0] preg_replace_callback_array()

The new preg_replace_callback_array() function enables code to be written more cleanly when using the preg_replace_callback() function. Prior to PHP 7, callbacks that needed to be executed per regular expression required the callback function to be polluted with lots of branching.

Now, callbacks can be registered to each regular expression using an associative array, where the key is a regular expression and the value is a callback.

[7.0] Integer division with intdiv()

The new intdiv() function performs an integer division of its operands and returns it.

var_dump(intdiv(3, 2));
var_dump(3/2);

int(1)
float(1.5)

PHP Major migration

Version
7.0 -> 7.1	http://php.net/manual/fr/migration71.php
5.6 -> 7.0	http://php.net/manual/fr/migration70.php
5.5 -> 5.6	http://php.net/manual/fr/migration56.php

Tips (There are micro optimizations)

[7] Packed arrays

Packed arrays is the first great PHP 7 optimization. Packed arrays consume less memory and are a bit faster in many operations than traditional arrays. A packed array must have the following characteristics:

The keys are only integers;

The keys are inserted into the array in growing only mode.

$a = ['foo', 'bar', 'baz'];
$a = [12 => 'baz', 42 => 'bar', 67 => [] ];

TIPS $pdo->query("SELECT * FROM table LIMIT 10000")->fetchAll(PDO::FETCH_NUM)

[7] ints/floats are free

In PHP 7, the way variables are allocated in memory changed dramatically. They went from being stored on the heap to stack memory pools. As a side effect, you may reuse variable containers for free: no memory allocated. That’s impossible to do in PHP 5, where each variable creation/assignment requires some memory allocation (which is bad for performance).

for ($i=0; $i<10000; $i++) {
    $$i = 'foo';
}

for ($i=0; $i<10000; $i++) {
    $$i = 42;
}

[7] References mismatch

Reference mismatch happens when you pass a non-ref variable to a function argument passed-by-ref, or the opposite. Something like this:

function foo(&$arg) { }
$var = 'str';
foo($var);

Do you know what horrible things happen under the hood in the PHP 5 engine? In PHP 5, whenever there is a reference mismatch, the engine is forced to duplicate the variable before passing it as an argument to a function. As you probably guessed, if the variable contains something big, like a several-thousands-entries array or a big chunk of string; the copy will take a lot of time.

This is because of the way variables and references used to work into PHP 5. When going into the body function, the engine doesn’t know yet if you will change the argument value. If you are, then as the argument is taken by reference, the change must be reflected outside if a reference variable were passed. What if it was not the case (as in our example)? The engine must then create a reference, from the non-reference variable you passed it in the function call. This procedure leads the engine to fully duplicate the variable content in PHP 5 (calling a lot of memcpy() on very sparse pointers, leading to tons of slow memory accesses).

In PHP 7, when the engine wants to create a reference from a non-reference variable, it just wraps the latter into the newly created former, that’s all. There is no memory copy, anywhere. This is because variables work in a very different way internally in PHP 7, and references have been deeply reworked.

Look at this piece of code:

function bar(&$a) { $f = $a; }

$var = range(1,1024);

for ($i=0; $i<1000; $i++) {
bar($var);
}

There is a double mismatch. When you call bar(), you force the engine to create a reference from $var to $a , as the signature says &$a. Then, now that $a is part of a reference set ($var-$a) into the bar() body, you affect it by value to $f: this is another mismatch. $f is not affected by reference, so $a-$f-$var must not be linked together, but $var-$a in one part, and $f alone in another part: you once more force the engine to perform some copies. In PHP 7, creating variables to/from references is pretty light, only copy-on-write may happen after that.

[7] Immutable arrays

Immutable arrays are a concept added to PHP 7, and part of the OPCache extension, which I assume you are using. An immutable array is an array full of immutable elements, which are compile-time known and fully resolved values: strings, integers, floats, or arrays containing those. In a word: no dynamic elements, no variables, like in this example:

$ar = [ 'foo', 42, 'bar', [1, 2, 3], 9.87 ];

Such arrays have been optimized in PHP 7. Under PHP 5, those are just plain arrays, there is no difference between an array containing dynamic members (such as $vars), and a static compile-time resolved array. In PHP 7, immutable arrays are not copied nor duplicated as soon as they stay read only.

Here is such an example:

for ($i = 0; $i < 1000; $i++) {
$var[] = [
  0 => 'Symfony\\Bundle\\FrameworkBundle\\EventListener\\SessionListener',
  1 => 'Symfony\\Component\\HttpFoundation\\Session\\Storage\\NativeSessionStorage',
  /* ... go to many immutable items here */
  ];
}

The above code creates the same big array 1,000 times (truncated, think of it as being a hundred or thousand slots array).

In PHP 5, this array is duplicated in memory 1,000 times. A big array could weight at several hundreds of Kilobytes or even Megabytes, which in this code will be duplicated 1,000 times. Such a waste of memory.

In PHP 7, OPCache flags those arrays as being immutable: the array is created only once, and shares its memory pointer everywhere it is supposed to be used, ending in huge savings of overall memory, especially when dealing with big configuration arrays, like the one shown in the example (which is taken from the Symfony 3 framework).

Let’s see the performance impact:

Illustrated example with composer : https://github.com/composer/composer/pull/5174