Pair program with me! profile for carousel at Stack Overflow, Q&A for professional and enthusiast programmers

4/27/2014

Laravel with OnApp cloud

Developing in (and for) the cloud has become defacto standard skill for any modern web developer. Without knowing how to, for example, setup Apache server on AWS Linux instance - important tools will be missing in your development toolset.
Many providers are moving to cloud. Fortrabbit (my provider of choice) is offering SSH connection to all application resources, so it is important to become familiar with cloud concepts.

These days I am working on one awesome project - it is basically Laravel SaaS application that connects to cloud servers, with the help of OnApp. OnApp is cloud management system for service providers.

There is no Laravel package for OnApp, but there is great  OnApp PHP wrapper, for API calls to virtual server. That wrapper is just an abstraction over php_curl, which internally handles all HTTP calls to servers, and returns response in one of two forms - XML of JSON.

Now, that PHP wrapper is very nicely designed, and allows easily communication with servers through native API. It has some very neat solutions, that makes pleasure working with API.

I will point some key features, that are crucial for understanding how it works.

First, to make use of wrapper, you will have to include it in your app. First line of code should be call to OnAppInit.php file, which autoloads all dependencies. Something like:

//based on my path
require "/lib/Onapp/OnAppInit.php";


With this line, all classes are autoloaded, and we can move to actual connection to server. This is where the beauty comes. Connection is made with the help of Factory class instance, which implements factory design patterns, that can instantiate all internal OnApp classes. Next, all OnApp classes inherits from base OnApp class, and allows unified interface for all inherited classes. It is done with the help of factory.

To connect to server, just call Factory class like this:


$factory = new OnApp_Factory($server,$user,$password);



Where $server represents full url to server (not just server.com), but requires protocol (http://server.com). If you made mistake, leaving out the protocol type, you can spend days trying to figure out where is the problem :)

After that you are connected, and ready to use API. But with basic connection, you can't go far. Factory class has factory method, that is responsible for instatiation of all other classes.
To make it more clear, here if that factory method (from the main Factory class):

public function factory( $name, $debug = false ) {
    $class_name = 'OnApp_' . $name;

    $result = new $class_name();
    $result->logger->setDebug( $debug );

    $result->setOption( ONAPP_OPTION_DEBUG_MODE, $debug );
    $result->logger->setTimezone();
    $result->version = $this->getAPIVersion();
    $result->options = $this->options;
    $result->_ch     = $this->_ch;
    $result->initFields( $this->getAPIVersion() );
    return $result;
}


Now things are more clear. Factory argument name will become OnApp clas name. Factory method also serves as configuration method, that will return new instance for chaining.

So to return User class, just call this:

$user = $factory->factory("User");

//or some other, useful classes
$user = $factory->factory("Log");

//or
$user = $factory->factory("BillingPlan");


Now, with those instances, you can use OnApp methods, that allows easily working with API. All instances have getList method. User class is used commonly (there have to be users :), so to list all users, you will do:

//list all server users
$user = $factory->factory("User")->getList($user_id);

// or to list all server logs
$user = $factory->factory("Log")->getList($user_id);


Remember that you are connected as user, with the help of base Factory class. Depending on you permisions (user of admin) you can list different users.

In a nutshell, this API allows creating powerfull SaaS applications, from your base code. In my case, all incoming data (in the form of JSON) are part of application, and I can manage users and server state easily from one central point. There is much more then this - you can pull statistics, work with hypervisors, work with virtual machines... It is really rich set of tools for cloud management.

Thanks for reading.

4/10/2014

How to use Xdebug profiler and Kcachegrind tools in PHP project?

I use more and more awesome Xdebug profiler in combination with Vdebug plugin and Kcachegrind.

Xdebug is standard PHP debugger. Vdebug is Vim plugin that let's you easily debug and trace PHP (and not just PHP) code from Vim. Kcachegrind is amazing piece of software which let's you visually represent whole application calling stack, and do a memory profile and discover bottlenecks.

You need some time and slight tweaks to enable all this tools to work.
But when you do that, you have awesome toolset that let's you dissect you code in a number of different ways. I mostly use it for learning new API-s, writing my extensions and looking at architecture and design of existing code (these days mostly Laravel).

Fist step is to install and enable xdebug. Here is how to do that on ubuntu:

    suto apt-get install php5-xdebug

And check in your php.ini that xdebug is enabled.

Here is my Xdebug setup from /etc/php5/apache2/php.ini file:

    zend_extension=/usr/lib/php5/20121212/xdebug.so
    xdebug.remote_enable=on
    xdebug.remote_handler=dbgp
    xdebug.remote_autostart=1
    xdebug.remote_host=localhost
    xdebug.remote_port=9000
    xdebug.profiler_output_dir="/var/log/xprofile/"
    xdebug.profiler_append=On
    xdebug.profiler_enable_trigger=On
    xdebug.profiler_output_name="%R-%u.trace"
    xdebug.trace_options=1
    xdebug.collect_params=4
    xdebug.collect_return=1
    xdebug.collect_vars=0  
    xdebug.profiler_enable=1
     ;xdebug.auto_trace=Off

As you can see it is divided into xdebug settings that let's you debug, and setting that allows profiling. Here you can find some advanced settings, which let's you save file names based on function call. By default profiler saves all files with some random prefix, and it is hard to distinguish between them

To install Vdebug on Vim, just install this plugin
https://github.com/joonty/vdebug and follow installation process. After that you wil be able to dubug and set breakpoints with builtin vim mappings.

In order to save and use output from profiles, you need to create default location for files. This is my location:

    xdebug.profiler_output_dir="/var/log/xprofile/"

Now, every time you start a project in the browser, a new profile file will be created.

Next tool is Kcachegrind. This is project home:

    http://kcachegrind.sourceforge.net/html/Home.html

But it can be easily installed through CLI of software center. Kcachegrind allows you to have visually insight into project calling stack. For me it was  far better learning experience then reading tons and tons of books. With this, I am actually "in the source code", without external explanation about that code. I can see how authors were designing whole application and have some advanced insight into whole process of app creation.

There is one gotcha with Xprofile. If you don't delete generated files, they can easily accumulate and consume lot of memory. You won't be always  using xdebug but profiler will always generate those files. So you need to manually delete them. It can be also done with some kind of deamon process and backround script.

This is how I solve this problem, with one command. I created alias like this:

     alias xd="sudo find /var/log/xprofile/* -mmin +0 -exec rm {} \;"

And placed it into my .zshrc file. With this alias command, all files inside xprofile directory older then 0 minutes will be deleted - which means all files will be deleted.

Alternatively, you can create new bash script/file, with this command:

    #!/usr/bin/sh
    sudo find /var/log/xprofile/* -mmin +0 -exec rm {} \;

Place it somewhere on your path, provide permissions and you will have globally executable command that will clean xprofile folder.

Thanks for reading.


3/14/2014

Laravel refactoring hell

Recently I had an opportunity as a freelancer, to work on existing codebase project. It was supposed to be a simple Laravel CRUD project, that does nothing more then collecting user data and storing it in a database. There were some people working on this before me, and I along with other members of the team supposed to finish that project.

Since I am primarly Jeffrey Way's Laravel student, I used to watch him how he writes clean and testable code. Also Jeffrey taught me to think very abstractly, and to think about future of the code - not just because of me, but because of other developers that could read my code.

I learned throughout  my career to teach programming just from high quality sources and to follow top level developers/programmers.

So in that sense (very naively) I was expecting to see everywhere that basic pattern, which is very human in its nature. Write and teach for people, not just for yourself. Code is a poetry - that is my moto. I am staring at this screen most of my day - so let me create art.

BUT

That code was everything else except human readable and maintanable.
Let me be more expressive - it was worst nightmare that can happen to a developer. It was big ball of mud. But very low quality mud - not the mud that you can wash easily - but stinky mud. Almost crap.

Ok, enough of pathetic. Lets talks about the facts. First let me point that project was old more that 5 months. Very trivial project that could be implemented for 2-3 weeks, was lasted for more then 5 months.
With potential to never ends in this form. While looking in the code, I became aware about the state of the mind of the prior developer. And his attitude toward this project. Everything was telling me that he hated this project and that he didn't have enough experience for that. When I was told that it was his first "bigger" Laravel project, everything becames clear to me.  It was natural for that project to became abandoned.
Also requirements were very poorly explained to me. There was no clear separation between developers in the team. Just make it work attitude. Project leaders were missing programming and leadership knowledge. It was obvious that this project needs refactoring and it is impossible to make it work. Well, it can just work,  but how long ? Every composer update can break whole project.

Can it be worse ?

I could stop writing this post because it is obvious what to expect next - pure horror written by unexperienced developer.

But there are some interesting things in that code. When I think now, it was rare opportunity for me to work on such a project. So I decided to learn as much as I can. 

Here are some juicy facts:

  • Prior developer was a kind of old-school PHP developer. He was trying to insert plain PHP as much as he can. 
  • He was forcing his own (bad) development style into Laravel architecture
  • He had his own rules about environment detection, completely ignoring Laravel environment handling ( or not knowing ).
  • Code is full of constants, even Laravel has some elegant solution for that. 
  • It is almost not commented.
  • There is no standard indentation, naming and code style consistency.
  • Powerfull blade templates are rarely used.
  • There are  view partials everywhere - without obvious reason.
  • There are views everywhere, with inline CSS, PHP, Javascript.
  • There is huge number of controllers, migrations, models... Everything is huge in this code. And everything is so ugly. He didn't know how to abstract things.
  • Pivot tables and complex relations are used with no reason, just to make things more complicated  
  • There is a lot of empty and not used functions/methods.
  • There is a front end code mixed with server side scripts (assets folder in app/). This was my favourite - when I mention that, I was cursed :)
  • There is a lot of everything everywhere, where it shouldn't be.
  • Dependencies, dependencies and dependencies everywhere.
  • Not tested at all - but should I expect it to be tested ? It is impossible to test this project.
How much should developer charge to refactor this ? I think most developers would write applicaton from scratch - it is easier.

And there are some valuable lessons learned from this. First I have to think about the people, not just about my ass. Someone can read that code for a month of six month. Next, what about responsibility ? Does this guy feel responsible for his work ? Does he think about the users of his project ? I don't think so. Does he think about his career ? No. Sadly, anyone can be self-declared Laravel developer. It is a big problem

That leads me to conclusion that the community should somehow consider Laravel certification. All big framework are certified - so why not Laravel ?

It is hard to find good clients. As a freelancer, I am forced to compete with kids that use dreamweawer and with clients that mostly dont know what they want. And to look at horror code, like this one.

Thanks for reading.

2/23/2014

My toolset

Developers have some special relationships with the tools  they use for everyday work.  I am not exception.

From the beginning of my carrier, I really spent lot of time trying different editors, terminals, operating systems... I was very passionate about that . Sometimes I didn't do anything, but just tuning new terminal or editor, which I quickly replaced with new one. I was aware that it could lead me to insoncistency,  so I was trying to learn as much as I can about new tools. By learning , I was sticking my  nose everywhere - from the Linux kernel to the most highest abstractions.

I didn't mentioned that Windows very quickly become boring, there was nothing interesting there for me. So I become dedicated Linux/Ubuntu user.  And Linux is perfect platform for any kind of programming (well not for C# but you know what I mean ). No developer can convince me that his favourite OS is Microsoft Windows, if  he is not their employee.

I was very fascinated (and still I am) with different (mostly open source) project. It is incredible how such masterpieces like Apache or Vim are totaly free, and you have support for any kind of problems - and it is still free.

These days I am not changing my tools so often - due to fact that I found my (almost) perfect  setup and that I don't have so much time.

So here is a quick overview what I use in my daily practice:

Main OS:
Linux/Ubuntu
I know, I know real hackers are using Gentoo and Slack but Ubuntu, which is based on Debian unstable is really up to date with bleeding edge packages - which are necessery due to fast changing web environment. Ubuntu 14.04. is going to be LTS (long term supported) so after upgrading, I wont be moving further for years.

Main editor:
Vim
What to say about Vim ? If you havent tried it, then no words can express what can Vim mean to productivity. Vim is iconic software and represents more then just text based editor. If you can't find something in the form of Vim extension, then you didn't look carefully. From Vim (with the help of terminal) I can code, debug, test, git, fetch...

Terminal emulators:
I am doing most of my work moving between browser and terminal.
I was using Terminator for long. Terminator has one great feature - you can split windows vertically and horizontally quickly. It is fully supported with all kind of character-sets and just works. One thing that I missed in Terminator was lack of line spacing. Only one Linux terminal that has that feature is Urxvt. Line spacing for me was so important, so I decided to migrate to Urxvt. It more geeky terminal, that needs some advanced configuration, but when you tune it by your needs - there is no turning back. But, nothing is perfect. On Ubuntu you can't set Urxvt transparent background  because Compiz has some weird issues with it. If you help me solve this, I'll buy you a beer , no matter in which part of the world you are :) Along with Urxvt comes Tmux, terminal multiplexer for managing multiple sessions.

Browsers:
Since I am primary a web developer, browser for me are just clients, with additional features. Chrome/Chromium team has produced awesome software. Chrome/Chromium developer tools is very powerful and if used correctly it can increase productivity, not just on client side. Chromium/Chrome is very opened toward developers and by just entering chrome://chrome-urls in the address bar, you can enjoy all different setting and statistics...

And that is it. There is so much power in everything above, that trying something else at this point would be just a waste of time.

At the end, I have to mention IDE-s. Only one that I was using for some time was KomodoEdit. It has good Linux support and some more good things  but that was all. I just don't know how to work in such environment. My fingers are in love with Vim keyboard bindings :)

Thanks for reading.

2/22/2014

Symfony in Laravel

Last couple of days i spent reading this tutorial. It is Fabien Potencier (creator of Symfony) online guide on how to create your own PHP framework on top of Symfony components. 

The reason I was reading this is because Laravel is heavily dependent on Symfony. I was trying to find out why Taylor Otwell (Laravel creator) choose Symfony to rely on.

I also watched some Fabien Potenciers video presentations and read some of his articles. I have to say, that this guy is not ordinary PHP geek, who is just a creator of one successful framework. There are some really great ideas in his talks and I highly recommend you to read something like this article. I was impressed wich some ideas like MVC is not a part of web, and Symfony doesn't have domain logic layer. If you thinks about this, there is a lot of truth inside. I found myself beeing amazed how much I learned from Symfony. 

So lets see what and why is Laravel using. The most important part of Symfony in Laravel is a HTTPKernelInterface. As a developer you know that PHP is interacting with server through SAPI (server application  programming interface). SAPI in PHP is basically implemented in the form of header and echo method. Through SAPI, PHP is sending its response to server. On Apache, default SAPI for PHP currently is Apache2handler. There is one more interface, for interacting with server - CLI SAPI or command-line interface. This is how PHP interact with server through command line. HTTPKernelInterface is build on top of idea that PHP needs better way to interact with server and clients. HTTPKernelInterface has just one method - handle. And that is all. Good programming practice suggest that interfaces should follow interface segregation principle. That principle states that no client should depend on the method that it doesn't use. So, that why there is only one method inside, that should provide all response logic.

Laravel is implementing this interface in the main Application class.There is a lot of things happening inside this implementation - from route collection lookup, to custom response. When you run app->run method, you are triggering stack that will end with returning response to client. Application has some bootstrap process before method run is triggered, but this method is the heart of Laravel HTTP handling logic.

It should be no surprise that both - Laravel request and response  are extensions of Symfony HTTPFoundation classes. So, Laravel is built on top of Symfony - I guess Taylor Otwell was first reading that tutorial :)

Web appllications are complex to develop, due to HTTP stateless nature.
On one end, we as developers have to take care about transport mechanism (protocol), and on another end we have to deal with domain logic (MVC). On native deskop applications, there is no HTTP. Everything is managed locally, and you can develop application in one consistent environment. I think that is the main reason for popularity of Single Page Web Applications. They are trying to simulate desktop environment, with moving most of logic to client. We will see how that trend is going to evolve.

Symfony ecosystem is huge. It is established PHP framework, so Laravel is gaining much of its strength from Symfony. So, if you want to learn more about Laravel, take some time and git clone Symfony. Besides weird yaml format (Laravel is using json) and some Symfony specific configurations, you can learn something new about architecture and design.

Thanks for reading. 



2/12/2014

SOA - Service Oriented Architecture in Laravel

My previous post title was n-tier meets MVC.
In this post I want to add one more thing to that conclusion - n-tier meets MVC with the help of SOA.

Yes, besides n-tier (client/server) and MVC our favourite framework is implementing one more well known architectural pattern. It is called SOA or Service Oriented Architecture. Wikipedia has got pretty good explanation of SOA.


Take a look at this sentence from Wiki:
Service provider: The service provider creates a web service and possibly publishes its interface and access information to the service registry.

Sounds familiar ? Well it took me a while untill I finally realized what is Laravel doing in one of its core parts. Like many Laravel developers I was struggling with facades, providers and containers terminology without knowing that it is just an implementation of well known pattern. Modified, but concept is still the same.

Less obvious part of this  is why Taylor Otwell actually decided to design framework with SOA style incorporeted ?
Knowing that he has strong .NET background can help. In Microsoft world, they traditionally care about services. I was teached to think Restfully, so this was hard concept to grasp.

So the story is:

In the middle of the Laravel  kingdom lives Application, that depends on its citizens and servants. Application can exists by its own, but there is no point in empty kingdom. Every kingom has a treasury.  So they build one. In this kingdom it is called Container . 
Then Application defined  a rules and exposed them publically through its main  accociate - ServiceProvider class. All new citizens have to behave and look the way it is defined by the king.
ServiceProvider announced that everyone who wants to be a part of Application kingdom must implements two methods:

  • register and
  • boot

Register to become part of kingdom and boot to invite their friends to join. Also, Application allows some level of freedom to their citizens and let them choose their names and how they will be called. Speeking in Laravel terms, they can use Aliases and Facades for their access.

But, not everything is perfect . Some citizens are on the higher position then others. So they are treated specially. e.g. Request class is called very early, and it is bound directly, without use of ServiceProvider class.
On the other hand some citizens choose fancy names like Config, but underneath they are originally called Repository.

What would be a kingdom without a police ? Events are everywhere.
They provide welcome to every new citizen (provider) and take care of illegal activities. If something bad happened they will fire its main weapon - Exception. In our kingdom it is allowed to have private police.
It is no problem. Even better - it is a free gift from one of the highly rated Applications servants. If you want to take care if right type of rules are applied in your Validation, just call Event police - and it will fire all exceptions.  

So, even it sometimes looks that things are out of order in typical Laravel application, it is just because developer didn't understood properly how framework is internally designed and with what intention in mind. In all other casess, we have great tool for building web applications.

Thanks for reading.

2/09/2014

n-tier meets MVC in Laravel


This is the post about two fundamental architectural patterns that can be found in most todays web framerworks. These patterns are:

  • n-tier represented as client/server model and
  • MVC (model,view,controller)

There is a lot resources online client/server, n-tier so I will take a look at Laravel specific implementations of these patterns. You can argue is MVC architectural or design pattern. I think MVC is closer to well established arch patterns due to its usage and overall complexity.

In Laravel we can exactly detect boundary between them as so as their main flow. Client model starts in the form of HTTP request, which is received from the browser (mostly) and provided to main Application class:

public function __construct(Request $request = null)
 {

  $this->registerBaseBindings($request ?: $this->createNewRequest());

  $this->registerBaseServiceProviders();

  $this->registerBaseMiddlewares();
 }


Request comes in a form of Laravel request which extends Symfony class. After application receives a request, it will registered it as a protected property (object). In this phase, flow is pretty straightforward. If there is no custom middleware functionality, request will end waiting for handler method. There are some minor processing like attaching session driver, but main job is done in a layer that comes after.

I mentioned middleware, which is actually all functionality that lives in a space between the reqeust hits the internal application routes. Laravel (from 4.1 Iversion) implements StackBuilder middleware, which can be used  like a flyweight alternative to caching and session management (and for some other things).

Here is a method that implements StackBuilder:

protected function getStackedClient()
 {
  $sessionReject = $this->bound('session.reject') ? $this['session.reject'] : null;

  $client = with(new \Stack\Builder)
      ->push('Illuminate\Cookie\Guard', $this['encrypter'])
      ->push('Illuminate\Cookie\Queue', $this['cookie'])
      ->push('Illuminate\Session\Middleware', $this['session'], $sessionReject);

  $this->mergeCustomMiddlewares($client);

  return $client->resolve($this);
 }


StackBuilder andLaravel Application class implements Symfony\HttpKernelInterface in the form of handle method. That is why it is called stack - it just forwards request further down the stack till it reaches main Application handler

When request hits Application handler, it is on the door of MVC part of Laravel, which leads further to domain logic and the heart of application.

Take a look at the main handler:

public function handle(SymfonyRequest $request, $type = HttpKernelInterface::MASTER_REQUEST, $catch = true)
 {
  try
  {
   $this->refreshRequest($request = Request::createFromBase($request));

   $this->boot();

   return $this->dispatch($request);
  }
  catch (\Exception $e)
  {
   if ($this->runningUnitTests()) throw $e;

   return $this['exception']->handleException($e);
  }
 }


As can be seen, it is dispatched to router, for further processing and which will result with HTTP response.   For now I wont go deeper into source code, since point of this post was just to show conceptual boundary between these two layers. This boundary is sometimes not so visible, due to frameworks specific implementations. One thing that is worth mentioning is that client/server model is dominant model in distributed systems like internet (where client is present), while MVC originated in the deskop environment. If not designed properly, applications made with the mix of these patterns can become very complex and lead to bugs and potential break. For me, it was very important to reckognize them in Laravel, so I could have better insight into awesome framework design.

Thanks for reading.