The dark art of PHP session management.

1. Hey look, a Magento Connect extension, let’s try it out

2. Oh, right, I need to be logged in to get the extension key

3. Where’s that password again? [knocks over pills]

4. Waiting for Magento site to log me in

5. [Still waiting]

There are, of course (I assume), reasons for Magento to force a sign-in before letting you grab a free extension key. That said, four plus years (!) in it’s still one of the things that annoys me about the platform. So here’s a bookmarklet that’ll extract the Connect 2.0 key from an extension page, even if you’re not logged in.

Visit the bookmarklet landing page, drag the Connect Key link to your bookmarks bar, click on it while ...]]> Stop me if this sounds familiar.

1. Hey look, a Magento Connect extension, let’s try it out

2. Oh, right, I need to be logged in to get the extension key

3. Where’s that password again? [knocks over pills]

4. Waiting for Magento site to log me in

5. [Still waiting]

There are, of course (I assume), reasons for Magento to force a sign-in before letting you grab a free extension key. That said, four plus years (!) in it’s still one of the things that annoys me about the platform. So here’s a bookmarklet that’ll extract the Connect 2.0 key from an extension page, even if you’re not logged in.

Visit the bookmarklet landing page, drag the Connect Key link to your bookmarks bar, click on it while you’re looking at a Magento Connect page, and a text field with a link will be appended at the top of the page. Tested in Chrome/OS X, should work elsewhere, bug reports welcome Your millage mary very, PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, etc.

You probably know all Magento admin console URLs need a nonce/key in the URL

http://magento.example.com/admin/catalog_product/edit/id/174/key/c4df66cd2118cb5422c9fb5eff7eq4f0/

That’s why we use the Mage::getModel('adminhtml/url') model object to generate URLs.

What I always forget is any POST to Magento’s backend also needs a form_key variable. Double the CSRF protection. Without this a POST will be redirected to the dashboard. You can generate a form_key with

Mage::getSingleton('core/session')->getFormKey()

Now that we’ve got a build environment up and running, we can get to work creating our first n98-magerun command. Our end goal for today is to add a helloworld command to n98-magrun

$ ./n98-magerun.phar list

//...

Available commands:
  helloworld                       Displays a Hello World message. (pedagogical

Development Stub

So far, all our examples have used the n98-magerun.phar bundled application. Therefore, our first step today is to run the application from source code. Every phar archive can have an optional “stub” file. This file is meant ...]]> This article is part of a longer series covering the n98-magerun power tool

Now that we’ve got a build environment up and running, we can get to work creating our first n98-magerun command. Our end goal for today is to add a helloworld command to n98-magrun

$ ./n98-magerun.phar list

//...

Available commands:
  helloworld                       Displays a Hello World message. (pedagogical

Development Stub

So far, all our examples have used the n98-magerun.phar bundled application. Therefore, our first step today is to run the application from source code. Every phar archive can have an optional “stub” file. This file is meant to encourage clean coding practices in your phar archive, with the stub acting as a main entry-point into your phar based application. Think of it like you would your index.php bootstrap file in a web application, or a main function in a c program.

The n98-magerun.phar bootstrap file is _cli_stub.php. You might think we could run this ourselves. However, running

$ php _cli_stub.php

will produce the error

PHP Fatal error:  Uncaught exception 'PharException' with message 'internal corruption of phar

Let’s look at the contents of the phar

#!/usr/bin/env php
<?php

Phar::mapPhar('n98-magerun.phar');

$application = require_once 'phar://n98-magerun.phar/src/bootstrap.php';
$application->setPharMode(true);
$application->run();

__HALT_COMPILER();

It turns out that phar stub files are meant to be used from phar archives only. The call to mapPhar, the require using a phar:// url scheme, and the __HALT_COMPILER() call all produce errors if used outside of a phar archive.

We’re going to need to create our own stub file. In the root of your repository, create the following file with the following contents

<!-- File: dev-stub.php -->
<?php
$application = require_once realpath(dirname(__FILE__)) . '/src/bootstrap.php';
$application->setPharMode(false);
$application->run();    

This is almost the same code that’s the the phar stub, but written with generic PHP.

Let’s try running this from the command line.

$ php dev-stub.php

     ___ ___
 _ _/ _ ( _ )___ _ __  __ _ __ _ ___ _ _ _  _ _ _
| ' \_, / _ \___| '  \/ _` / _` / -_) '_| || | ' \
|_||_/_/\___/   |_|_|_\__,_\__, \___|_|  \_,_|_||_|
                           |___/
n98-magerun version 1.63.0 by netz98 new media GmbH

Usage:
  [options] command [arguments]

...

Eureka! We’re now running n98-magerun strictly from code. You can make sure everything’s working correctly by navigating to your Magento folder and running a simple command like sys:info.

$ cd /path/to/magento
$ php /path/to/n98-magerun/dev-stub.php sys:info

  Magento System Information  

Version                  : 1.7.0.1
Edition                  : Community
Cache Backend            : Zend_Cache_Backend_File
Cache Directory          : /Users/alanstorm/Sites2012/magento1point7pointzeropoint1.dev/var/cache
Session                  : files
Crypt Key                : 18cea96f92f35a540d83a2fe7f33c005
Install Date             : Sun, 01 Jul 2012 17:06:09 +0000

We’re now ready to start with our helloworld command.

Hello World

To create a new n98-magerun command there’s five basic steps

1. Register the command

2. Create the command class

3. Configure the command name

4. Implement the command

5. Build our new command into the phar

By the end of the article, you’ll know how to do all five.

Registering the Command Class and Namespaces

Our first task is to register the command class. Open up the main application file and look at the registerCommands method

#File: src/N98/Magento/Application.php
//...    
class Application extends BaseApplication
{    
    //...
    protected function registerCommands()
    {
        $this->add(new GenerateLocalXmlConfigCommand());
        $this->add(new DatabaseDumpCommand());
        $this->add(new DatabaseDropCommand());        
        //...
    }
    //...    
}    

The n98-magerun team made use of Symfony’s console package as the basis for their application. A Symfony console application’s add method allows client programmers to tell their application about a command, which in turn exposes it to end users of the console application.

A few of you may be wondering why this code uses such generic class names. The name Application, and BaseApplication seem like things that could be used in other code bases — so why did n98-magerun choose Application and why did Symfony choose BaseApplication?

The short answer is namespaces. If you look at the top of this file

#File: src/N98/Magento/Application.php
namespace N98\Magento;
//...

you’ll see we’re working in the N98\Magerun namespace. That means this application class’s full name is actually

N98\Magento\Application

Similarly, further down in the file, you can see (nestled in among many similar calls)

use Symfony\Component\Console\Application as BaseApplication;

This aliases the class with the full name of

Symfony\Component\Console\Application

as the class BaseApplication in this file.

PHP namespace programming can be confusing if you’re not used to it. Even if you are used to it, the lack of a gold standard in how to approach namespaces can leave you feeling as though you’re flailing about. While this isn’t necessarily a namespace tutorial, we’ll try to explain how and why each classes interacts in this brave new namespaced world.

The namespace detour completed, let’s add our command! Add the following line to the registerCommands method.

#File: src/N98/Magento/Application.php
//...    
class Application extends BaseApplication
{    
    //...
    protected function registerCommands()
    {
        $this->add(
            new HelloWorldCommand()
        );
        //...
    }
}

What we’ve done here is pass in an object, instantiated from the class HelloWorldCommand. Let’s run our command with this in place

$ php /path/to/n98-magerun/dev-stub.php    
PHP Fatal error:  Class 'N98\Magento\HelloWorldCommand' not found    

A fatal error! PHP is complaining it can’t find a class named HelloWorldCommand. Actually, it’s complaining it can’t find a class named N98\Magento\HelloWorldCommand since we’re working in the N98\Magento namespace. Regardless, this make sense since we haven’t actually created our class yet. Let’s get to that.

Creating the Class

Each command in n98-magerun corresponds to a single PHP class file. These files are, by convention, created in the src/N98/Magento/Command folder.

If you list out the contents of this folder, you’ll see a number of classes and directories

$ ls -1 src/N98/Magento/Command
AbstractMagentoCommand.php
AbstractMagentoStoreConfigCommand.php
Admin
Cache
Cms
Config
ConfigurationLoader.php
Customer
Database
Design
Developer
Indexer
Installer
LocalConfig
MagentoConnect
OpenBrowserCommand.php
PHPUnit
ScriptCommand.php
SelfUpdateCommand.php
ShellCommand.php
System

We’re going to create a new class file in this folder. The n98-magerun autoloader expects file names and class names to match, so create the following file at the following location

#File: src/N98/Magento/Command/HelloWorldCommand.php
<?php
namespace N98\Magento\Command;    
use N98\Magento\Command\AbstractMagentoCommand;

class HelloWorldCommand extends AbstractMagentoCommand
{
}    

The first line of this file puts us in the

N98\Magento\Command

namespace. That means the full name for the HelloWorldCommand class we’ve declared is actually N98\Magento\Command\HelloWorldCommand. You’ll notice the namespace convention matches the file path convention.

N98/Magento/Command/HelloWorldCommand.php
N98\Magento\Command\HelloWorldCommand

Without getting too deeply into it, the autoloader depends on this, so make sure you stick to this convention.

Our HelloWorldCommand class extends the AbstractMagentoCommand class. This is the abstract class that all n98-magerun commands inherit from. The full name of this class is actually

N98\Magento\Command\AbstractMagentoCommand

We’re able to refer to it as AbstractMagentoCommand since the following code was used at the top of our file

use N98\Magento\Command\AbstractMagentoCommand;

If we run our application with the above class in place

$ php /path/to/dev-stub.php
Class 'N98\Magento\HelloWorldCommand' not found

We’re still getting the same error. That’s because our register function is trying to load a class in the current namespace of the Application file. If we change our add method call so it uses the full class name

#File: src/N98/Magento/Application.php
//...    
class Application extends BaseApplication
{    
    //...
    protected function registerCommands()
    {
        $this->add(
            new \N98\Magento\Command\HelloWorldCommand()
        );
    }
}

we should be good to go. Notice the leading \\ on the class name. This tells PHP to start looking from the global namespace, (vs. looking for a deeper namespace, starting at the current one) .

If we run our stub with the above in place

$ php /path/to/dev-stub.php
PHP Fatal error:  Uncaught exception 'LogicException' with message 'The command name cannot be empty.

Sort of Eureka! We’ve gotten rid of the class not found error, and replaced with with a new “the command name cannot be empty” error. That’s progress of a sort, and brings us to step 3, configure the command name.

Configuring the Command

Every command has a configure method which is called automatically. This is where we’ll want to assign our command a name. Add the following method to your command class

#File: src/N98/Magento/Command/HelloWorldCommand.php
<?php
namespace N98\Magento\Command;    
use N98\Magento\Command\AbstractMagentoCommand;

class HelloWorldCommand extends AbstractMagentoCommand
{
    protected function configure()
    {
        $this->setName('helloworld');

    }    
}    

With the above in place, lets run our command again

$ php /path/to/dev-stub.php
     ___ ___
 _ _/ _ ( _ )___ _ __  __ _ __ _ ___ _ _ _  _ _ _
| ' \_, / _ \___| '  \/ _` / _` / -_) '_| || | ' \
|_||_/_/\___/   |_|_|_\__,_\__, \___|_|  \_,_|_||_|
                           |___/
n98-magerun version 1.63.0 by netz98 new media GmbH

Usage:
  [options] command [arguments]

Options:
  --help           -h Display this help message.
  --quiet          -q Do not output any message.
  --verbose        -v Increase verbosity of messages.
  --version        -V Display this application version.
  --ansi              Force ANSI output.
  --no-ansi           Disable ANSI output.
  --no-interaction -n Do not ask any interactive question.

Available commands:
  helloworld                       
  help                             Displays help for a command

Finally! We’ve got a clean run. Also, if you’re playing close attention, you’ll notice that helloworld is now listed as an available command.

Available commands:
  helloworld                       
  help                             Displays help for a command

If you want to give your command a description, just call the setDescription method in configure

#File: src/N98/Magento/Command/HelloWorldCommand.php
protected function configure()
{
    $this->setName('helloworld');
    $this->setDescription('Displays a Hello World message. <comment>(pedagogical)</comment>');
}

Give the stub another run, and you should see your command description.

$ php /path/to/dev-stub.php
...
Available commands:
  helloworld                       Displays a Hello World message. (pedagogical)

Notice that the text we included in the <comment/> tag has been automatically made into a parenthetical.

Implementing our Command

We’ve got our stub running clean again — lets press our luck and try calling our helloworld command.

$ php /path/to/dev-stub.php helloworld

  [LogicException]                                                       
  You must override the execute() method in the concrete command class.  

Drats! At least we’ we’ve replaced our gross PHP errors with pretty Symfony errors.

Every command in a Symfony console application needs an execute method. The execute method is where we implement our command logic. To create this method, add the following to your HelloWorldCommand class

#File: src/N98/Magento/Command/HelloWorldCommand.php
writeln("Hello World!");
        }        
    }    
    
As you can see, we've added an `execute` method to our class.  The method has two arguments.  The first, `$input`, allows you to access arguments and options passed to your command.  The second, `$output`, allows you to send feedback to the users of your command.  You can also see we've called the `writeln` method of the output object to print out our ubiquitous *Hello World!* message.  Let's give the command a final run and call it a day.

    $ php /path/to/dev-stub.php helloworld
    PHP Catchable fatal error:  Argument 1 passed to N98\Magento\Command\HelloWorldCommand::execute() must be an instance of N98\Magento\Command\InputInterface, instance of Symfony\Component\Console\Input\ArgvInput given    
    
Crud.  That would have been too easy, wouldn't it have?  PHP is complaining that the first argument to `execute` failed to match the type safety check.  PHP was expecting a `N98\Magento\Command\InputInterface`, but was provided with a `Symfony\Component\Console\Input\ArgvInput`.  Again, namespaces rear their hydra like heads.  

The `InputInterface` interface we want is actually `Symfony\Component\Console\Input\InputInterface`.  Instead of changing the type check in front of the parameter, let's import `Symfony\Component\Console\Input\InputInterface` and `Symfony\Component\Console\Output\OutputInterface` into the current namespace.  Add the following lines just below the `namespace` declaration.

    #File: src/N98/Magento/Command/HelloWorldCommand.php
    namespace N98\Magento\Command;
    
    use Symfony\Component\Console\Input\InputInterface;
    use Symfony\Component\Console\Output\OutputInterface;    
    //...
    
With the above in place, we should be able to run our command.

    $ php ~/Documents/github_netz98/n98-magerun/dev-stub.php helloworld
    Hello World!

There we have it, another *Hello World* example program added to the world. 

Following Conventions
--------------------------------------------------
Before we finish things off by compiling our new code into a `phar`, let's jump back to our `add` method call.

    #File: src/N98/Magento/Application.php
    //...
    $this->add(
        new \N98\Magento\Command\HelloWorldCommand()
    );
    
    $this->add(new GenerateLocalXmlConfigCommand());    
    
While this code works, it's not keeping with the conventions setup by the netz98 team.  When they add a command object, they're not using the full qualified namespace, they're just using simple class names like `GenerateLocalXmlConfigCommand`.  How do they get away with this?

If you search this file for `GenerateLocalXmlConfigCommand`, you'll find your answer.  Near the top of the file is the following line
    
    #File: src/N98/Magento/Application.php
    use N98\Magento\Command\LocalConfig\GenerateCommand as GenerateLocalXmlConfigCommand;    
    
Using the `use` operator, the fully qualified `N98\Magento\Command\LocalConfig\GenerateCommand` class is being imported into the local namespace as `GenerateLocalXmlConfigCommand`.  Let's bring **our** class into the local namespace. Add the following right above the `GenerateLocalXmlConfigCommand` line

#File: src/N98/Magento/Application.php
//our code
use N98\Magento\Command\HelloWorldCommand;

//their code
use N98\Magento\Command\LocalConfig\GenerateCommand as GenerateLocalXmlConfigCommand;    

and then change your call to add

$this->add(new HelloWorldCommand());

Your command should behave exactly the same, but now you’re matching the current coding conventions used in the project. Some people might write this off as bike shedding, but a consistant codebase helps with team cohesion, and allows new programmers a stable base to learn the fundamentals of a project without having to worry about parsing multiple different coding styles. This is particularly true in the PHP world, since “idiomatic PHP” varies from project to project.

Building a New phar

The only thing left is to build a new phar archive. As we learned last time, our build script is just a single call to

$ phing
...
BUILD FINISHED

Total time: 8.2973 seconds away.  

After running phing, make sure a new phar was created with today’s date and time (May XX XX:XX below)

$ ls -l n98-magerun.phar 
-rwxrwxr-x  1 alanstorm  staff  3617497 May XX XX:XX n98-magerun.phar

Then, try running your command

$ ./n98-magerun.phar helloworld
Hello World!

Congratulations, you’ve just created and built your first n98-magerun command.

All the XML in code blocks is gone. Vanished. Something something other people’s free blogging systems.

Fortunately the post is mostly available via the V1 API (with entity encoded HTML), but that’s still some pretty weak sauce. I for one do not welcome our always available in the cloud closed source applications.

Today we’re going to run through setting up your own n98-magerun development environment. This will allow you to develop your own features and contribute bug fixes back to the main repository. This one’s a little longer than our normal Magento Quickies fare, so if you’re in a hurry you may want to save this for later.

There’s two (or maybe three) extra software packages you’ll need to install if you want to work with the n98-magerun source code. The first is PHP Composer, the second is the phing build system.

Composer

Composer is a software package for managing project dependencies. The n98-magerun program relies ...]]> This article is part of a longer series covering the n98-magerun power tool

Today we’re going to run through setting up your own n98-magerun development environment. This will allow you to develop your own features and contribute bug fixes back to the main repository. This one’s a little longer than our normal Magento Quickies fare, so if you’re in a hurry you may want to save this for later.

There’s two (or maybe three) extra software packages you’ll need to install if you want to work with the n98-magerun source code. The first is PHP Composer, the second is the phing build system.

Composer

Composer is a software package for managing project dependencies. The n98-magerun program relies on many third party packages for its functionality. Rather than include these in the main source tree, n98-magerun includes a top level vendor/ folder. This is where third party dependencies are downloaded to.

This is a standard best practice in profesional software development. By separating your vendor packages from the main source tree, you make upgrading the packages easier. You also encourage/force a “no hacking the vendor package” discipline on your team.

There’s instructions for installing Composer on their website. The short version is there’s a composer.phar (similar to n98-magerun.phar) archive that needs to be in your shell path and needs to be executable. The composer team has a “download install script with curl and run it” one liner, or you may download the phar file directly and place it in your path.

You can test if composer is installed correctly by running the following

$ composer.phar --version
Composer version 0209bd31a0ac3aeb2a68fc81e2d03c71072bef33

Phing

Phing is a PHP build system based on apache ant. For The Kids™ in the audience, ant is a system that was created to give developers a “pure java” build system (as opposed to the powerful-but-tortuous options of the day like GNU make). Phing, in turn, is a PHP version of ant.

Phing implements an XML based build language that allows you to perform common build tasks. This XML format is an example of a Domain Specific Language (DSL). Folks using Magento’s layout system may get a sense of Déjà vu.

Installing phing is slightly trickier than installing PHP composer. The preferred way of installing phing is via PHP PEAR. PEAR (the third package we mentioned above) is PHP’s original dependencies and package management system. While powerful, it never became consistently adopted by the disperate PHP communities. A side effect of this is many PHP distributions don’t have PEAR properly installed, or have it only available to users with sudo permission.

If a fully operational PEAR isn’t available to you, phing is also available via a package download, as well as a stand alone phar file.

Regardless how you install it, at the end of the day you’ll have a phing (or phing.phar or phing-latest.phar) command executable from your shell.

$ phing -v
Phing 2.5.0

Downloading Dependencies with Composer

Now that our tools are in place, we’re ready to start. Step 1 for working with n98-magerun is to get the codebase from GitHub.

$ git clone https://github.com/netz98/n98-magerun.git

However, even after you’ve cloned the git source tree, you’re still missing a number of files n98-magerun needs to function. If you look in the vendor folder, you’ll see it’s empty

$ ls -la vendor/
total 16
drwxr-xr-x   3 alanstorm  staff   102 Apr 26 14:40 .
drwxr-xr-x  26 alanstorm  staff   884 Apr 25 23:24 ..

As mentioned earlier, the vendor folder is where third party code libraries are kept. To download these files, run the composer.phar install command from the root folder of your repository.

$ cd n98-magerun
$ composer.phar install
Loading composer repositories with package information
Installing dependencies from lock file
  - Installing symfony/process (v2.2.1)
    Loading from cache

  - Installing symfony/finder (v2.2.1)
    Loading from cache

  - Installing symfony/console (2.2.x-dev a10a2fb)
    Cloning a10a2fb156267b9b8f01d717a06979e6ebb69251

  - Installing seld/jsonlint (1.1.1)
    Loading from cache

  - Installing justinrainbow/json-schema (1.1.0)
    Loading from cache

  - Installing composer/composer (dev-master c655674)
    Cloning c65567447c84eb6a8c0f229c40b0865cd4dcecb5

  - Installing fzaninotto/faker (v1.1.0)
    Loading from cache

  - Installing n98/junit-xml (dev-master 6381433)
    Cloning 6381433a39498dc22bec60cebdb2576a9d2327b0

  - Installing symfony/translation (v2.2.1)
    Loading from cache

  - Installing symfony/validator (v2.2.1)
    Loading from cache

  - Installing symfony/yaml (v2.2.1)
    Loading from cache

symfony/translation suggests installing symfony/config (2.2.*)
symfony/validator suggests installing doctrine/common (~2.2)
symfony/validator suggests installing symfony/config (2.2.*)
symfony/validator suggests installing symfony/http-foundation (2.2.*)
symfony/validator suggests installing symfony/locale (2.2.*)
Generating autoload files

If you take a look at your vendor folder after running the install command, you’ll see the required vendor packages have been installed.

$ ls -1a vendor/
.
..    
autoload.php
bin
composer
fzaninotto
justinrainbow
n98
seld
symfony

Of course, this raises a few questions: How does composer.phar know which packages to download? And where are these packages coming from? While this knowledge isn’t strictly required to procede, we’re going to cover it to put your (and our) minds at ease.

The composer.phar command looks for a file named composer.json. This file contains configuration information for a composer based project (which n98-magerun is). This configuration information will include a list of package dependencies

//File: composer.json
"require": {
    "php": ">=5.3.3",
    "symfony/console": "2.2.x-dev",
    "symfony/finder": "2.2.1",
    "symfony/yaml": "2.2.1",
    "symfony/process": "2.2.1",
    "symfony/validator": "2.2.1",
    "justinrainbow/json-schema": "1.1.*",
    "seld/jsonlint": "1.*",
    "n98/junit-xml": "dev-master",
    "fzaninotto/faker": "1.1.0",
    "composer/composer": "dev-master"
},

As you can see, the require key contains a list of packages and versions. These package names correspond with the downloaded packages from the install command. So this answers the “how does composer know” question. Before we move on the “where” question, it’s worth pointing out that composer is actually reading from the composer.lock file. This composer.lock file is generated with the information in composer.json, and is not meant to be human editable. If you want to get into the nitty gritty of this checkout the composer manual.

The “from where” question is a little trickier. There’s two grand arcs to the composer project. The first is the composer.phar application we downloaded, which contains the logic for dependency management. The second is a composer repository at packagist.org. Unless a different repository is listed in composer.json, packagist.org is where composer.phar will grab package information from.

Does that mean composer is downloading packages from packagist.org? No. Confused? Read on!

A composer repository only contains meta-data about a package. This meta-data includes a source code repository where the package source can be found. The packagist.org repository simply tells composer.phar where it should download the package source from.

Figuring out where composer.phar is grabbing packages from is a two step process. First, you can list out all the packages used in your project with the show command and and --self flag.

$ composer.phar show --self
name     : n98/magerun
descrip. : Tools for managing Magento projects and installations
keywords : magento, installer
versions : * dev-master
type     : library
license  : MIT
source   : []  
dist     : []  
names    : n98/magerun

support
issues : https://github.com/netz98/n98-magerun/issues

autoload
psr-0
N98 => src
Composer => src
Xanido => src

requires
php >=5.3.3
symfony/console 2.2.x-dev
symfony/finder 2.2.1
symfony/yaml 2.2.1
symfony/process 2.2.1
symfony/validator 2.2.1
justinrainbow/json-schema 1.1.*
seld/jsonlint 1.*
n98/junit-xml dev-master
fzaninotto/faker 1.1.0
composer/composer dev-master

requires (dev)
phpunit/phpunit 3.7.19
phpunit/phpunit-story 1.0.2
phpunit/phpunit-selenium 1.2.12
phpunit/dbunit 1.2.3
phpunit/php-invoker 1.1.2

Once you have a package name (such as symfony/yaml), you can see its repository by using the show command again, this time with the package name as an argument.

$ composer.phar show symfony/yaml
name     : symfony/yaml
descrip. : Symfony Yaml Component
keywords : 
versions : dev-master, 2.3.x-dev, 2.2.x-dev, * v2.2.1, v2.2.0, v2.2.0-RC3, v2.2.0-RC2, v2.2.0-RC1, v2.2.0-BETA2, v2.2.0-BETA1, 2.1.x-dev, v2.1.9, v2.1.8, v2.1.7, v2.1.6, v2.1.5, v2.1.4, v2.1.3, v2.1.2, v2.1.1, v2.1.0, v2.1.0-RC2, v2.1.0-RC1, v2.1.0-BETA4, v2.1.0-BETA3, v2.1.0-BETA2, v2.1.0-BETA1, 2.0.x-dev, v2.0.23, v2.0.22, v2.0.21, v2.0.20, v2.0.19, v2.0.18, v2.0.17, v2.0.16, v2.0.15, v2.0.14, v2.0.13, v2.0.12, v2.0.10, v2.0.9, 2.0.7, 2.0.6, 2.0.5, 2.0.4
type     : library
license  : MIT
source   : [git] https://github.com/symfony/Yaml.git v2.2.1
dist     : [zip] https://api.github.com/repos/symfony/Yaml/zipball/v2.2.1 v2.2.1
names    : symfony/yaml

autoload
psr-0
Symfony\Component\Yaml\ => .

requires
php >=5.3.3

The two sections we’re interested in here are source and dist. The source lists the repository where the source code should be downloaded from, (git, svn, and hg repository types are supported). This is how composer.phar will grab the needed vendor packages.

The archive (as opposed to repository) at dist can be used to install a package instead, but only if the --prefer-dist flag is used with the install command.

If you’re paranoid about security, you’ll want to review these package locations — although at some point using a distributed packaging system like CPAN, ruby gems, or packagist requires a certain level of trust. That, however, is another subject better left to another time.

Composer Development Packages

There’s one last thing we should cover before moving on from composer. If you examined the composer.json file carefully, you may have noticed this section

//File: composer.json
"require-dev": {
    "phpunit/phpunit": "3.7.19",
    "phpunit/phpunit-story": "1.0.2",
    "phpunit/phpunit-selenium": "1.2.12",
    "phpunit/dbunit": "1.2.3",
    "phpunit/php-invoker": "1.1.2"
},

This appears to be a second list of packages. It turns out composer projects can have two sets of required packages. The first is the standard distribution you’ll need to use the package. The second, require-dev, is an additional set of packages you’ll need to work on (or develop) the project.

Since this whole article is about building our own version of n98-magerun.phar, we’ll want to install these development packages. Let’s run composer.phar with the --dev flag.

$ composer.phar install --dev
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for phpunit/php-invoker 1.1.2 -> satisfiable by phpunit/php-invoker 1.1.2.
    - phpunit/php-invoker 1.1.2 requires ext-pcntl * -> the requested PHP extension pcntl is missing from your system.

Uh oh. At this point, you may (as we did above) run into an intractable problem. The phpunit/php-invoker package couldn’t be installed because it requires that PHP have the process control (ext-pcntl) extension installed. This extension is omitted from many standard PHP distributions.

We’ve now run into what I like to call the Software is Hard problem. In order to install these dependencies, we’d need to recompile our version of PHP. That’s something many people can’t, or won’t do. Unless we’re willing to make a massive change to our environment, we won’t be able to install the full suite of required phpunit tools.

On the flip side, we could remove phpunit/php-invoker as a requirement, and then successfully install the other development packages. However, this would mean we’re veering off from what the main team is doing. Our system may start to behave differently than theirs, causing subtle and hard to track down compatibility problems in the future.

There’s going to be times in your career where you run into problems where there is no ideal solution. The only option is to raise your concerns, and then press on with the less than ideal conditions. The trick to a long career is finding a way to not let these small defeats sap your motivation for other, more important tasks.

Fortunately, it’s still possible to work on n98-magerun code without the full phpunit suite, so we’ll carry on without it.

Phing

Now that we’ve discussed composer, our next step is using phing to build the project. Take a look at the build.xml file in the root of the project

<!-- File: build.xml -->
<project name="n98-magerun" default="dist">

    <fileset dir="." id="root_folder"><!-- ... --></fileset>
    <fileset dir="src"  id="src_folder"><!-- ... --></fileset>
    <fileset dir="res" id="res_folder"><!-- ... --></fileset>
    <fileset dir="vendor" id="vendor_folder"><!-- ... --></fileset>

    <target name="dist">
        <!-- ... -->
    </target>

    <target name="dist_unix">
        <!-- ... -->
    </target>

    <target name="install">
        <!-- ... -->
    </target>

</project>

This is our phing build file. It contains all the build instructions for our package. We’re interested in the <target/> nodes. Each <target/> node defines a single build command. For example, the install command can be used to install the n98-magerun.phar from the repository to your local computer. This is a relatively simple build target.

<target name="install">
    <exec command="sudo cp ${project.basedir}/n98-magerun.phar /usr/local/bin/n98-magerun.phar;" />
    <exec command="sudo chmod a+x /usr/local/bin/n98-magerun.phar;" />
</target>

As you can see, there’s two <exec/> nodes within this target. These are command nodes — think of each one like you would a single line in a normal programming language or shell script. The <exec/> command is one of the simplest build commands — it simply runs a unix shell script.

Let’s try running the install target.

$ phing install
Buildfile: /path/to/n98-magerun/build.xml

n98-magerun > install:

Password:
Sorry, try again.
Password:

BUILD FINISHED

Total time: 5.9880 seconds

As you can see, to run a phing command you simply pass it the target name. The phing command will look for a build.xml file in the current directory, and then run the target you’ve passed in.

As for the command output itself, it prompts you for a password (since it’s running the sudo command), and the finishes the build. Unfortunately, there’s no indication of what the command actually did. If you’d like a little more feedback, try the verbose flag.

$ phing -verbose install
Buildfile: /path/to/n98-magerun/build.xml
Override ignored for user property phing.file
Override ignored for user property phing.dir
parsing buildfile build.xml
Project base dir set to: /path/to/n98-magerun
Build sequence for target 'install' is: install 
Complete build sequence is: install dist dist_unix 

n98-magerun > install:

Property ${project.basedir} => /path/to/n98-magerun
     [exec] Executing command: sudo cp /path/to/n98-magerun/n98-magerun.phar /usr/local/bin/n98-magerun.phar; 2>&1
     [exec] Executing command: sudo chmod a+x /usr/local/bin/n98-magerun.phar; 2>&1

BUILD FINISHED

Total time: 0.1772 seconds

With the verbose flag, phing tells you everything it’s doing. From this we can figure out a file has been copied to /usr/local/bin/n98-magerun.phar. This places the phar in our *nix path, and is another way to install the n98-magerun.phar command.

Building the Project

Most phing build files have a default target which builds the project.

If you run phing with the -l option

$ phing -l
Buildfile: /path/to/n98-magerun/build.xml
Default target:
-------------------------------------------------------------------------------
 dist

Subtargets:
-------------------------------------------------------------------------------
 dist
 dist_unix
 install

you’ll get a list of each target in the application, as well as the default target (in this case, dist). You can also see the default target by looking at the root node of the build.xml file.

<!-- build.xml -->
<project name="n98-magerun" default="dist">

We’re going to run through a default build of the n98-magerun tool, including several problems we ran into and how to work around them. You may not run into these specific problems — and you may run into different problems not mentioned here. Setting up a build environment is always a trial and error process, and things will rarely work out of the gate. Part of the reason teams use a tool like phing is to provide a single entry point into their build system and force developers to download and install the same dependencies. If you run into trouble getting this up and running, the Magento Stack Exchange is a great place to ask questions to help get your system up and running.

To run the default build step, just run phing from the project root directory with no arguments

$ phing

The first thing you’ll see is phing going through the project dependencies and downloading the needed files to the vendor folder.

$ phing
Buildfile: /path/to/n98-magerun/build.xml

n98-magerun > dist:

Loading composer repositories with package information
Installing dependencies from lock file
  - Installing symfony/process (v2.2.1)
    Loading from cache

  - Installing symfony/finder (v2.2.1)
    Loading from cache

... snipped ...

If you’ve already downloaded these files with the composer.phar install command, these downloads will be skipped. The output for that looks like this

$ phing
Buildfile: /private/tmp/building/n98-magerun/build.xml

n98-magerun > dist:

Loading composer repositories with package information
Installing dependencies from lock file
Nothing to install or update
Generating autoload files

If you look at the build.xml file, you can see all this happens because the command in the dist target runs the composer install command.

<target name="dist">
    <exec command="composer.phar install" dir="${project.basedir}" passthru="true" />

So, it would be more accurate to say phing still attempts to update your dependencies, but if the latest versions are already downloaded composer will skip re-downloading them.

Writeable PHAR Archives

After phing runs the composer.phar command, it moves on to the next build steps, and (for some of you) will hit another error.

Execution of target “dist_unix” failed for the following reason: /path/to/n98-magerun/build.xml:45:65: Problem creating package: creating archive “/path/to/n98-magerun/n98-magerun.phar” disabled by the php.ini setting phar.readonly

Here we’ve discovered a constraint of PHP’s phar implementation. By default, PHP won’t let you edit an existing phar archive. While annoying, if you consider that a phar archive is meant to “emulate jars and dlls”, this makes sense. Allowing a program to modify its libraries would be a recipe for disaster.

The problem, of course, is that it’s PHP code that creates a phar archive. In order to solve this chicken/egg problem, PHP has a phar.readonly flag for php.ini. If you set this flag to 0 or Off, PHP will allow you to access the methods which modify a phar archive.

This flag is one you’ll need to set in your php.ini file, it can’t be set at runtime with ini_set. If you’re not sure where your php.ini file is located, try running the following

$ php --ini
Configuration File (php.ini) Path: /etc
Loaded Configuration File:         /private/etc/php.ini
Scan for additional .ini files in: (none)
Additional .ini files parsed:      (none)

Remember, it’s not uncommon for the command line version of PHP to have a different ini file than the web server version of PHP.

BZip Compression Problems

With the above in place, try running the build again. If you’re like me, you’re about to run into another problem

Execution of target “dist_unix” failed for the following reason: /path/to/n98-magerun/build.xml:45:65: Problem creating package: unable to create temporary file [wrapped: unable to create temporary file]

This one is a head scratcher, but I finally tracked down the problem. In short: While PHP’s phar format works across a variety of platforms and environments, the methods for creating phar archives are still very beta-ish. The specific problem here is, when compressing phar archives PHP likes to open a temporary file — and from the sound of things it opens a temporary file for each file in the archive, and doesn’t close them until the command is complete. All that library code means n98-magerun has over 1,000 files.

This quickly hits the limits that a non-server operating system has in place for the number of files each process is allowed to have open. To solve this problem, you’ll need to edit build.xml. Find the following XML node

<!-- File: build.xml -->
<pharpackage basedir="./" stub="_cli_stub.php" signature="sha512" compression="bzip2" destfile="./n98-magerun.phar">    

The pharpackage node is a phing build command that allows you to automatically create a phar application from a PHP codebase. We’re interested in the compression="bzip2" attribute. This tells phing to run the phar archive through the compressFiles method, and is done to reduce the file size of the archive that’s distributed. If you change this attribute to

compression="none"

then phing will skip the compression step, and we should be able to get a complete build.

$ phing
Buildfile: /path/to/n98-magerun/build.xml

n98-magerun > dist:

Loading composer repositories with package information
Installing dependencies from lock file
Nothing to install or update
Generating autoload files
[phingcall] Calling Buildfile '/path/to/n98-magerun/build.xml' with target 'dist_unix'

n98-magerun > dist_unix:

[pharpackage] Building package: /path/to/n98-magerun/n98-magerun.phar
    [chmod] Changed file mode on '/path/to/n98-magerun/n98-magerun.phar' to 775
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file

BUILD FINISHED

Total time: 3.5711 seconds

That looks like a clean build, let’s test out the n98-magerun.phar file that was just created.

$ ./n98-magerun.phar --version
n98-magerun version 1.63.0 by netz98 new media GmbH

Success! However, our n98-magerun.phar file

$ ls -lh n98-magerun.phar
-rwxrwxr-x  1 alanstorm  staff   3.4M Apr 25 15:50 n98-magerun.phar

is over 3 MB in size. Compare that to the 1 MB file distributed in the main github archive. So, while we can’t build exactly what the core n98-magerun team does, we can get a clean build in place with a testable phar, which is good enough for our needs.

While frustrating, you’ll find many points in your career where your development environment won’t match up exactly with that of the team you’re working with. While it’s true (especially among non-distributed teams) that a chaotic build environment can lead to development delays, these delays are subtle and complicated in nature, and not the sort of thing that shows up in ROI metrics. Unfair as it may be, as an individual developer working with a group, it will often fall on you to solve these problems for yourself in a way that doesn’t disrupt team flow.

Wrap Up

Regardless of the subtle incompatibilities, we’ve successfully built the n98-magerun.phar file from source. This means we can start hacking on bugs, as well as adding new features to the tool. Next time we’ll create our very first n98-magerun command.

And Vinai’s fix for the duplicate URL problem partially/previously mentioned below. I try not to be hypercritical of Magento Inc./eBay because I know how hard these things are, but it really sounds like they pushed EE 1.13 out the door to have a Magento Imagine announcement. Fortunately the partner firms are there to catch these things and make smart choices for their clients.

Some musings by Fabrizio Branca on changes to Magento’s product URL handling. I haven’t dug deeply into this, but it sounds like there’s a new typed (vs. a varchar), per store EAV attribute for product URL keys with unique indexing, but the unique indexing hasn’t been thought through completely. Hopefully this is fixed up before the 1.8 CE alpha becomes the shipping version.

The database: That which, as Magento programmers, we’re not supposed to touch. Use the native objects is a refrain I’ve sung on more than one occasion, and it’s still what I recommend to people starting out with Magento.

Of course, even if we treat the database as a black box datastore, we still need to move that block box datastore around. Also, once you understand the database schemas for a particular Magento sub-system, its often easier to work directly with the database for read operations.

For those intrepid developers willing to brave the wilds of Magento’s database, n98-magerun has a suite of database commands to make ...]]> This article is part of a longer series covering the n98-magerun power tool

The database: That which, as Magento programmers, we’re not supposed to touch. Use the native objects is a refrain I’ve sung on more than one occasion, and it’s still what I recommend to people starting out with Magento.

Of course, even if we treat the database as a black box datastore, we still need to move that block box datastore around. Also, once you understand the database schemas for a particular Magento sub-system, its often easier to work directly with the database for read operations.

For those intrepid developers willing to brave the wilds of Magento’s database, n98-magerun has a suite of database commands to make things easier.

$ n98-magerun list db

Available commands for the "db" namespace:
  db:console   Opens mysql client by database config from local.xml
  db:drop      Drop current database
  db:dump      Dumps database with mysqldump cli client according to informations from local.xml
  db:import    Imports database with mysql cli client according to database defined in local.xml
  db:info      Dumps database informations
  db:query     Executes an SQL query on the database defined in local.xml

First, the db:info command allows you to extract useful information from Magento’s database configuration.

$ n98-magerun db:info
host                     : localhost
username                 : root
password                 : ididit27
dbname                   : magento_example_com
initStatements           : SET NAMES utf8
model                    : mysql4
type                     : pdo_mysql
pdoType                  : 
active                   : 1
PDO-Connection-String    : mysql:host=localhost;dbname=magento_example_com
JDBC-Connection-String   : jdbc:mysql://localhost/magento_example_com?username=root&password=ididit27
MySQL-Cli-String         : mysql -h'localhost' -u'root' -p'ididit27' 'magento_example_com'

Of particular interest are the PDO-Connection-String, JDBC-Connection-String, and MySQL-Cli-String fields. If you work with external systems that need to connect to Magento’s database, then having these strings prebuilt can save you from all sorts of tedious typo debugging. The db:info command is also a nice sanity check for the inevitable “am I editing the right database” problem.

Taking this laziness a step further, if you’re just looking to connect to Magento’s mysql database via the command line, then the db:console command is just what you’re looking for.

$ n98-magerun db:console
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 12052
Server version: 5.5.29 MySQL Community Server (GPL)

Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Similarly, the db-dump command will export your entire mysql database to a text file.

$ n98-magerun db:dump

    Dump MySQL Database  

Filename for SQL dump: [magento_example_com.sql]
Start dumping database magento_example_com to file magento_example_com.sql
Finished

The db-dump command also has a useful “strip” feature. This allows you to omit INSERTs from your dump file for specific tables, dumping only the structure. For example, if you wanted to export a complete Magento system, but omit all the static blocks (the cms_block and cms_block_store tables), you’d do something like this.

$ n98-magerun db:dump --strip="cms_block cms_block_store"

While useful, this still requires you understand what data can and can’t be dumped. Omit the wrong join table and you might break your entire Magento system. To help with this problem, the n98-magerun authors have created special “table groups” that logically group related tables together. For example, you can strip out customer data using the @customers group with the following

$ n98-magerun db:dump --strip="@customers"

  Dump MySQL Database  

Filename for SQL dump: [magento_example_com.sql]
No-data export for: customer_address_entity customer_address_entity_datetime customer_address_entity_decimal customer_address_entity_int customer_address_entity_text customer_address_entity_varchar customer_entity customer_entity_datetime customer_entity_decimal customer_entity_int customer_entity_text customer_entity_varchar
Start dumping database magento_example_com to file magento_example_com.sql
Finished

Notice the list of tables listed after No-data export for. These are the tables whose data has been stripped. You can see a list of the table groups by using the built-in help

$ n98-magerun help db:dump 
...
Available Table Groups
 @log Log tables
 @dataflowtemp Temporary tables of the dataflow import/export tool
 @stripped Standard definition for a stripped dump (logs and dataflow)
 @sales Sales data (orders, invoices, creditmemos etc)
 @customers Customer data
 @trade Current trade data (customers and orders). You usally do not want those in developer systems.
 @development Removes logs and trade data so developers do not have to work with real customer data

Keep in mind — these groups are defined by the creators of n98-magerun. It’s still a good idea to review the dumped data and ensure the information you want stripped is stripped. Even if the n98-magerun list of tables matches up with your own mental model, there’s no way this tool could know about 3rd-party extensions that may be saving sensitive data in other tables.

Once you’ve dumped your data, you’ll probably want to import it at some point. The db:import command does exactly what you’d think. Just pass in the path of your sql file as the first argument and n98-magerun will import the file.

$ n98-magerun db:import magento_example_com.sql

Another useful option supported by both db:import and db-dump is --only-command. You can use this option to spit out the command line that n98-magerun is building to perform the dump/import.

mysql -h'localhost' -u'root' -p'ididit27' 'magento_example_com' < 'magento_example_com.sql'

There’s also a command, db:query, for running raw SQL queries against the database, with results returned as tab separated lines

n98-magerun db:query "SELECT * FROM cms_page"
page_id title   root_template   meta_keywords   meta_description    identifier  content_heading content creation_time   update_time is_active   sort_order  layout_update_xml   custom_theme    custom_root_template    custom_layout_update_xml    custom_theme_from   custom_theme_to
12  404 Not Found 1 two_columns_right   Page keywords   Page description    no-route    NULL    <div class="page-head-alt"><h3>Whoops, our bad...</h3></div>\n\n<dl>\n\n<dt>The page you requested was not found, and we have a fine guess why.</dt>\n\n<dd>\n\n<ul class="disc">\n\n<li>If you typed the URL directly, please make sure the spelling is correct.</li>\n\n<li>If you clicked on a link to get here, the link is outdated.</li>\n\n</ul></dd>\n\n</dl>\n\n<br/>\n\n<dl>\n\n<dt>What can you do?</dt>\n\n<dd>Have no fear, help is near! There are many ways you can get back on track with Magento Demo Store.</dd>\n\n<dd>\n\n<ul class="disc">\n\n<li><a href="#" onclick="history.go(-1);">Go back</a> to the previous page.</li>\n\n<li>Use the search bar at the top of the page to search for your products.</li>\n\n<li>Follow these links to get you back on track!<br/><a href="{{store url=""}}">Store Home</a><br/><a href="{{store url="customer/account"}}">My Account</a></li></ul></dd></dl><br/>\n\n<p><img src="{{skin url='images/media/404_callout1.jpg'}}" style="margin-right:15px;"/><img src="{{skin url='images/media/404_callout2.jpg'}}" /></p>    2007-06-20 18:38:32 2013-03-21 05:04:55 1   0   NULL    NULL    NULL    NULL    NULL    NULL
13  Home page   two_columns_right   NULL    NULL    home    NULL    <div class="col-left side-col">\n\n<p class="home-callout">\n\n<a href="{{store direct_url="apparel/shoes/womens/anashria-womens-premier-leather-sandal.html"}}"><img src="{{skin url='images/ph_callout_left_top.gif'}}" border="0" /></a>\n\n</p>\n\n<p class="home-callout">\n\n<img src="{{skin url='images/ph_callout_left_rebel.jpg'}}" border="0" />\n\n</p>\n\n{{block type="tag/popular" template="tag/popular.phtml"}}\n\n</div>\n\n<div class="home-spot">\n\n<p class="home-callout">\n\n<img src="{{skin url='images/home_main_callout.jpg'}}" border="0" width="470" />\n\n</p>\n\n<p class="home-callout">\n\n<img src="{{skin url='images/free_shipping_callout.jpg'}}" border="0" width="470" />\n\n</p>\n\n<div class="box best-selling">\n\n<h3>Best Selling Products</h3>\n\n<table border="0" cellspacing="0">\n\n\t<tbody>\n\n\t\t<tr class="odd">\n\n\t\t\t<td><a href="{{store direct_url="sony-vaio-vgn-txn27n-b-11-1-notebook-pc.html"}}"><img class="product-img" src="{{skin url='images/media/best_selling_img01.jpg'}}" border="0" width="95" /></a>\n\n\t\t\t<div class="product-description">\n\n\t\t\t<p><a href="{{store direct_url="sony-vaio-vgn-txn27n-b-11-1-notebook-pc.html"}}">Sony VAIO VGN-TXN27N/B 11.1" Notebook PC</a></p>\n\n\t\t\t<p>\n\n\t\t\tSee all <a href="{{store direct_url="electronics/computers/laptops.html"}}">Laptops</a>\n\n\t\t\t</p>\n\n\t\t\t</div>\n\n\t\t\t</td>\n\n\t\t\t<td><a href="{{store direct_url="nine-west-women-s-lucero-pump.html"}}"><img class="product-img" src="{{skin url='images/media/best_selling_img02.jpg'}}" border="0" width="95" /></a>\n\n\t\t\t<div class="product-description">\n\n\t\t\t<p><a href="{{store direct_url="nine-west-women-s-lucero-pump.html"}}">Nine West Women's Lucero Pump</a></p>\n\n\t\t\t<p>\n\n\t\t\tSee all <a href="{{store direct_url="apparel/shoes.html"}}">Shoes</a>\n\n\t\t\t</p>\n\n\t\t\t</div>\n\n\t\t\t</td>\n\n\t\t</tr>\n\n\t\t<tr class="even">\n\n\t\t\t<td><a href="{{store direct_url="olympus-stylus-750-7-1mp-digital-camera.html"}}"><img class="product-img" src="{{skin url='images/media/best_selling_img03.jpg'}}" border="0" width="95" /></a>\n\n\t\t\t<div class="product-description">\n\n\t\t\t<p>\n\n\t\t\t<a href="{{store direct_url="olympus-stylus-750-7-1mp-digital-camera.html"}}">Olympus Stylus 750 7.1MP Digital Camera</a>\n\n\t\t\t</p>\n\n\t\t\t<p>\n\n\t\t\tSee all <a href="{{store direct_url="electronics/cameras/digital-cameras.html"}}">Digital Cameras</a>\n\n\t\t\t</p>\n\n\t\t\t</div>\n\n\t\t\t</td>\n\n\t\t\t<td><a href="{{store direct_url="acer-ferrari-3200-notebook-computer-pc.html"}}"><img class="product-img" src="{{skin url='images/media/best_selling_img04.jpg'}}" border="0" width="95" /></a>\n\n\t\t\t<div class="product-description">\n\n\t\t\t<p>\n\n\t\t\t<a href="{{store direct_url="acer-ferrari-3200-notebook-computer-pc.html"}}">Acer Ferrari 3200 Notebook Computer PC</a>\n\n\t\t\t</p>\n\n\t\t\t<p>\n\n\t\t\tSee all <a href="{{store direct_url="electronics/computers/laptops.html"}}">Laptops</a>\n\n\t\t\t</p>\n\n\t\t\t</div>\n\n\t\t\t</td>\n\n\t\t</tr>\n\n\t\t<tr class="odd">\n\n\t\t\t<td><a href="{{store direct_url="asics-men-s-gel-kayano-xii.html"}}"><img class="product-img" src="{{skin url='images/media/best_selling_img05.jpg'}}" border="0" width="95" /></a>\n\n\t\t\t<div class="product-description">\n\n\t\t\t<p><a href="{{store direct_url="asics-men-s-gel-kayano-xii.html"}}">ASICS&reg; Men's GEL-Kayano&reg; XII</a></p>\n\n\t\t\t<p>See all <a href="{{store direct_url="apparel/shoes.html"}}">Shoes</a></p>\n\n\t\t\t</div>\n\n\t\t\t</td>\n\n\t\t\t<td><a href="{{store direct_url="coalesce-functioning-on-impatience-t-shirt.html"}}"><img class="product-img" src="{{skin url='images/media/best_selling_img06.jpg'}}" border="0" width="95" /></a>\n\n\t\t\t<div class="product-description">\n\n\t\t\t<p>\n\n\t\t\t<a href="{{store direct_url="coalesce-functioning-on-impatience-t-shirt.html"}}">Coalesce: Functioning On Impatience T-Shirt</a>\n\n\t\t\t</p>\n\n\t\t\t<p>\n\n\t\t\tSee all <a href="{{store direct_url="apparel/shirts.html"}}">Shirts</a>\n\n\t\t\t</p>\n\n\t\t\t</div>\n\n\t\t\t</td>\n\n\t\t</tr>\n\n\t</tbody>\n\n</table>\n\n</div>\n\n</div>   2007-08-23 10:03:25 2013-03-21 05:04:55 1   0   <!--<reference name="content">\n\n<block type="catalog/product_new" name="home.catalog.product.new" alias="product_new" template="catalog/product/new.phtml" after="cms_page"><action method="addPriceBlockType"><type>bundle</type><block>bundle/catalog_product_price</block><template>bundle/catalog/product/price.phtml</template></action></block>\n\n<block type="reports/product_viewed" name="home.reports.product.viewed" alias="product_viewed" template="reports/home_product_viewed.phtml" after="product_new"><action method="addPriceBlockType"><type>bundle</type><block>bundle/catalog_product_price</block><template>bundle/catalog/product/price.phtml</template></action></block>\n\n<block type="reports/product_compared" name="home.reports.product.compared" template="reports/home_product_compared.phtml" after="product_viewed"><action method="addPriceBlockType"><type>bundle</type><block>bundle/catalog_product_price</block><template>bundle/catalog/product/price.phtml</template></action></block>\n\n</reference><reference name="right">\n\n<action method="unsetChild"><alias>right.reports.product.viewed</alias></action>\n\n<action method="unsetChild"><alias>right.reports.product.compared</alias></action>\n\n</reference>-->    NULL    NULL    NULL    NULL    NULL
...

Again, this query is run using the standard mysql client tool — if you want the command line string use the --only-command flag.

$ n98-magerun db:query --only-command "SELECT * FROM cms_page" 
mysql -h'localhost' -u'root' -p'ididit27' 'magento_example_com' -e 'SELECT * FROM cms_page'

Finally, a db:drop command is included as well, allowing you to drop the entire database. Given my bias against having such destructive commands at your finger tips, I’ll leave this one as an exercise for the reader.

In time of investigation I saw on forums that other developers around have similar issue that wasn’t resolved yet. After tracing and looking what might went wrong we’ve found that… If you’re interested in ...]]> Recently one of our clients had contacted us and he said that ProductAlert functionality doesn’t work any more. After I investigated the situation I saw that last email regarding to Stock Alerts was sent before several months. In a meanwhile we’ve upgraded site on Magento EE and I firstly thought that maybe during the upgrade something went wrong. Other thought was that maybe client has modified Transactional Email Template… After reviewing a log files I couldn’t find anything related to those emails. Other emails system does send.

In time of investigation I saw on forums that other developers around have similar issue that wasn’t resolved yet. After tracing and looking what might went wrong we’ve found that… If you’re interested in solution keep reading.

Shortly, issue was in Magento approach how they handle Product Stock Alert collection. Our client has more than 40 000 subscribers and more than 30 000 are waiting for status to be changed from “Out of Stock” to “In Stock”. So when Magento tries to load 40 000 records from table product_alert_stock to collection to foreach them because of memory limitation php fails. Probably if you have more than 20 000 records in table product_alert_stock your Product Stock Alert functionality doesn’t work anymore.

Solution was relatively simple. Divide collection by 1000 and create several pages to iterate through more than 30 000 recirds, in same way as Magento is doing in different places in it’s system. In our case we had around 35 pages, each with 1000 records in collection. New functionality is iterating through all of them and for each record where status=0 (unprocessed) system checks for product to see if product is now in stock. If that’s true => set status=1 and send an notification email to customer.

So let’s create our module, Inchoo_ProductAlert, that will enhance Magento’s approach:

1. Create file in app/etc/modules/ Inchoo_ProductAlert.xml with the following content:

<?xml version="1.0"?>
<config>
    <modules>
        <Inchoo_ProductAlert>
            <active>true</active>
            <codePool>local</codePool>
        </Inchoo_ProductAlert>
    </modules>
</config>

2. Create file in app/code/local/Inchoo/ProductAlert/etc/ config.xml with the following content:

<?xml version="1.0"?>
<config>
    <modules>
        <Inchoo_ProductAlert>
            <version>1.0.0.0</version>
        </Inchoo_ProductAlert>
    </modules>
    <global>
        <models>
            <productalert>
                <rewrite>
                    <observer>Inchoo_ProductAlert_Model_Observer</observer>
                </rewrite>
            </productalert>
        </models>
    </global>
</config>

You can see that we’ve rewritten Mage_ProductAlert_Model_Observer class

3. Create file in /www/app/code/local/Inchoo/ProductAlert/Model/ Observer.php with the following content:

<?php
/**
 * ProductAlert observer
 *
 * @category   Inchoo
 * @package    Inchoo_ProductAlert
 * @author     <ivan.galambos@inchoo.net>
 */
class Inchoo_ProductAlert_Model_Observer extends Mage_ProductAlert_Model_Observer
{
     /**
     * Process stock emails
     *
     * @param Mage_ProductAlert_Model_Email $email
     * @return Mage_ProductAlert_Model_Observer
     */
    protected function _processStock(Mage_ProductAlert_Model_Email $email)
    {
        $email->setType('stock');
        foreach ($this->_getWebsites() as $website) {
            /* @var $website Mage_Core_Model_Website */
            if (!$website->getDefaultGroup() || !$website->getDefaultGroup()->getDefaultStore()) {
                continue;
            }
            if (!Mage::getStoreConfig(
                self::XML_PATH_STOCK_ALLOW,
                $website->getDefaultGroup()->getDefaultStore()->getId()
            )) {
                continue;
            }
            try {
                $wholeCollection = Mage::getModel('productalert/stock')
                    ->getCollection()
//                    ->addWebsiteFilter($website->getId())
                    ->addFieldToFilter('website_id', $website->getId())
                    ->addFieldToFilter('status', 0)
                ;
//                $wholeCollection->getSelect()->order('alert_stock_id DESC');
                /*       table: !product_alert_stock!
                alert_stock_id: 1
                   customer_id: 1
                    product_id: 1
                    website_id: 1
                      add_date: 2013-04-26 12:08:30
                     send_date: 2013-04-26 12:28:16
                    send_count: 2
                        status: 1
                */
            }
            catch (Exception $e) {
                Mage::log('error-1-collection $e=' . $e->getMessage(), false, 'product_alert_stock_error.log', true);
                $this->_errors[] = $e->getMessage();
                return $this;
            }
            $previousCustomer = null;
            $email->setWebsite($website);
            try {
                $originalCollection = $wholeCollection;
                $count = null;
                $page  = 1;
                $lPage = null;
                $break = false;
                while ($break !== true) {
                    $collection = clone $originalCollection;
                    $collection->setPageSize(1000);
                    $collection->setCurPage($page);
                    $collection->load();
                    if (is_null($count)) {
                        $count = $collection->getSize();
                        $lPage = $collection->getLastPageNumber();
                    }
                    if ($lPage == $page) {
                        $break = true;
                    }
                    Mage::log('page=' . $page, false, 'check_page_count.log', true);
                    Mage::log('collection=' . (string)$collection->getSelect(), false, 'check_page_count.log', true);
                    $page ++;
                    foreach ($collection as $alert) {
                        try {
                            if (!$previousCustomer || $previousCustomer->getId() != $alert->getCustomerId()) {
                                $customer = Mage::getModel('customer/customer')->load($alert->getCustomerId());
                                if ($previousCustomer) {
                                    $email->send();
                                }
                                if (!$customer) {
                                    continue;
                                }
                                $previousCustomer = $customer;
                                $email->clean();
                                $email->setCustomer($customer);
                            }
                            else {
                                $customer = $previousCustomer;
                            }
                            $product = Mage::getModel('catalog/product')
                                ->setStoreId($website->getDefaultStore()->getId())
                                ->load($alert->getProductId());
                            /* @var $product Mage_Catalog_Model_Product */
                            if (!$product) {
                                continue;
                            }
                            $product->setCustomerGroupId($customer->getGroupId());
                            if ($product->isSalable()) {
                                $email->addStockProduct($product);
                                $alert->setSendDate(Mage::getModel('core/date')->gmtDate());
                                $alert->setSendCount($alert->getSendCount() + 1);
                                $alert->setStatus(1);
                                $alert->save();
                            }
                        }
                        catch (Exception $e) {
                            Mage::log('error-2-alert $e=' . $e->getMessage(), false, 'product_alert_stock_error.log', true);
                            $this->_errors[] = $e->getMessage();
                        }
                    }
                }
                Mage::log("\n\n", false, 'check_page_count.log', true);
            } catch (Exception $e) {
                Mage::log('error-3-steps $e=' . $e->getMessage(), false, 'product_alert_stock_error.log', true);
            }
            if ($previousCustomer) {
                try {
                    $email->send();
                }
                catch (Exception $e) {
                    $this->_errors[] = $e->getMessage();
                }
            }
        }
        return $this;
    }
    /**
     * Run process send product alerts
     *
     * @return Inchoo_ProductAlert_Model_Observer
     */
    public function process()
    {
        Mage::log('ProductAlert started @' . now(), false, 'product_alert_workflow.log', true);
        $email = Mage::getModel('productalert/email');
        /* @var $email Mage_ProductAlert_Model_Email */
        $this->_processPrice($email);
        $this->_processStock($email);
        $this->_sendErrorEmail();
        Mage::log('ProductAlert finished @' . now(), false, 'product_alert_workflow.log', true);
        return $this;
    }
}

You can see that we’ve overwritten 2 methods: process() and _processStock().
In process() method we added Mage::log() for creating start&end time in var/log/product_alert_workflow.log so we can know did script finish with it’s execution. Similarly to debug our enhancement in _processStock() method, we added few more Mage:log() calls so we can track if everything behave in the expected way.

For the end, if you don’t want to wait for your cron to be triggered, you can create a file in Magento root, let’s call it,

inchoo_cron.php:

<?php
/**
 * @author     <ivan.galambos@inchoo.net>
 */
require_once 'app/Mage.php';
Mage::app();
try {
    Mage::getModel('productalert/observer')->process();
} catch (Exception $e) {
    Mage::log('error-0-start $e=' . $e->getMessage() . ' @' . now(), false, 'product_alert_stock_error.log', true);
}

And that’s about it. Feel free after testing your functionality to remove/comment Mage::log() calls. And if you have similar issue with Product Alert Price functinality you can apply same idea and everything should work fine.

3 log files are going to be created if you apply this enhancement without removing Mage::log() calls:

• product_alert_workflow.log – log start&end time for each call to script
• check_page_count.log – log #of pages and query that should be run
• product_alert_stock_error.log – error messages from 0-3 places in code

That’s about it.

Have a nice day!

We’re almost at the end of our n98-magerun command review. Today we’re going to blitz through the remaining commands on our plate

$ n98-magerun list local
    Available commands for the "local-config:generate" namespace:
      local-config:generate   Generates local.xml config

$ n98-magerun list index
    Available commands for the "index" namespace:
      index:list          Lists all magento indexes
      index:reindex       Reindex a magento index by code
      index:reindex:all   Reindex all magento ...]]>
				This article is part of a longer series covering the n98-magerun power tool

We’re almost at the end of our n98-magerun command review.  Today we’re going to blitz through the remaining commands on our plate
$ n98-magerun list local
    Available commands for the "local-config:generate" namespace:
      local-config:generate   Generates local.xml config

$ n98-magerun list index
    Available commands for the "index" namespace:
      index:list          Lists all magento indexes
      index:reindex       Reindex a magento index by code
      index:reindex:all   Reindex all magento indexes

$ n98-magerun list extension
    Available commands for the "extension" namespace:
      extension:download   Download magento-connect package
      extension:install    Install magento-connect package
      extension:list       List magento connection extensions
      extension:search     List magento connection extensions
      extension:upgrade    Upgrade magento-connect package

Let’s get to it.
Local Config
The local-config:generate command will recreate the local configuration file at app/etc/local.xml.
$ n98-magerun local-config:generate

  Generate Magento local.xml  

Please enter the database host:localhost
Please enter the database username:merlin
Please enter the database password:password12345
Please enter the database name:magento_example_com
Please enter the session save:file
Please enter the admin frontname:admin
Generated config

This file is the first one loaded into the global configuration tree, and is generated by installing Magento. This command appears to exist as a fix for stores with a missing local.xml.  Consider the results if you run the command with app/etc/local.xml in place.
$ n98-magerun local-config:generate
local.xml file already exists in folder "/path/to/magento/app/etc"

Also, local-config:generate relies on the existence of the app/etc/local.xml.template file that ships with Magento.  Running the command without this will result in the following error.
File /Users/alanstorm/Sites2012/magento1point7pointzeropoint1.dev/app/etc/local.xml.template does not exist.

Finally, while there are interactive prompts for entering data, the script does no validation on the values.  Database credentials aren’t checked, it’s possible to enter an invalid session save type, etc.  Keep your wits about you when using this command.
Indexer Commands
The index category contains three commands for working with Magento’s indexes.  These commands are very similar to the shell/indexer.php script that ships with Magento.  For those new to the platform, a Magento “index” is conceptually like an index on a database column, but a more practical definition might be
> An index takes data Magento either stores across many tables in a normalized-(ish) way or calculates, and then de-normalizes or caches it for performance and convenience reasons.
You can use the index:list command to fetch a list of all known Magento indexers, (also known as “index processes”).
$ n98-magerun index:list

  Index list  

+---------------------------+-----------------+--------+
| code                      | status          | time   |
+---------------------------+-----------------+--------+
| catalog_product_attribute | pending         | 0      |
| catalog_product_price     | pending         | 2m 52s |
| catalog_url               | pending         | 5s     |
| catalog_product_flat      | pending         | 1m 27s |
| catalog_category_flat     | require_reindex | 1s     |
| catalog_category_product  | pending         | 0      |
| catalogsearch_fulltext    | pending         | 3s     |
| cataloginventory_stock    | pending         | 1s     |
| tag_summary               | pending         | 0      |
+---------------------------+-----------------+--------+

The status column indicates if Magento thinks a re-index is required (pending being the normal status), and the time column indicates how long the last re-index took.
You can run a specific index process with the index:reindex command
$ n98-magerun index:reindex
$ n98-magerun index:reindex catalog_url

  Reindex  

Started reindex of: catalog_url
Estimated end: 2013-04-23 04:41:27 UTC
Successfully reindexed catalog_url (Runtime: 5s)

or re-index everything with the index:reindex:all command.
$ n98-magerun index:reindex:all
Successfully reindexed catalog_product_attribute
Successfully reindexed catalog_product_price
Successfully reindexed catalog_url
Successfully reindexed catalog_product_flat
Successfully reindexed catalog_category_flat
Successfully reindexed catalog_category_product
Successfully reindexed catalogsearch_fulltext
Successfully reindexed cataloginventory_stock
Successfully reindexed tag_summary

Magento Connect Commands
The extension category contains commands related to Magento Connect.  Magento Connect is the repository of free Magento extensions maintained by eBay Inc.’s Magento division, as well as the ecosystem of tools for installing the extension packages.
While the extension commands may seem like a replacement for the bundled ./mage script, these commands are  just wrappers for calls to ./mage.  You’ll still need a fully installed and configured Magento Connect to run these commands.
The extension:list command will display a list of every Magento Connect extension that’s available.  As of this writing that’s 3800 extensions.
$ mn98-magerun extension:list 
+---------------------+--------------------+--------+
| Package             | Version            | Stabi  |
+---------------------+--------------------+--------+
| Lib_Js_Calendar     | 1.51.1.1           | stable |
| Lib_Js_Ext          | 1.7.0.0            | stable |
| ... snipped ...     |                    |        |
| Ivan_ImageZoom      | 2.0.0              | stable |
| fullpagecache       | 1.0.0              | stable |
+---------------------+--------------------+--------+

If you’re looking for a specific extension, you can perform a simple stristr search on the name with the extension:search command.
$ mn98-magerun extension:search Mage

+-------------------------+----------------+--------+
| Package                 | Version        | Stabi  |
+-------------------------+----------------+--------+
| Mage_Pear_Helpers       | 1.0.18800      | stable |
| Mage_All                | 1.0.19870      | stable |
| Mage_Oscommerce         | 1.1.20123      | dev    |
| Mage_Checkout           | 1.0            | dev    |
| Mage_Strikeiron         | 1.0.18800      | stable |
| Locale_Mage_Core_fr_FR  | 1.0.18800      | beta   |
| ... snipped ...         |                |        |
| mageokcustomoptions     | 1.0.0.0        | alpha  |
| Ivan_ImageZoom          | 2.0.0          | stable |
+-------------------------+----------------+--------+

Keep in mind stristr’s search is not a sophisticated algorithm. You’ll notice our search returned Ivan_ImageZoom due to the mage in Image.
Once you’ve found the extension you’re looking for, you may download it with the extension:download command
$ n98-magerun extension:download Pulsestorm_Launcher
Saved to: /path/to/magento/downloader/.cache/community/Pulsestorm_Launcher-1.0.3.tgz

You can also download and automatically install a package with extension:install.
$ n98-magerun extension:install Pulsestorm_Launcher
Checking dependencies of packages
Starting to download Pulsestorm_Launcher-1.0.3.tgz ...
...done: 8,946 bytes
Installing package community/Pulsestorm_Launcher 1.0.3
Package community/Pulsestorm_Launcher 1.0.3 installed successfully
Package installed: community/Pulsestorm_Launcher 1.0.3

And finally, you can use the extension:upgrade command to check-for-then-automatically-install an upgrade for an extension.
$ n98-magerun extension:upgrade Pulsestorm_Launcher
Checking dependencies of packages
Already installed: community/Pulsestorm_Launcher 1.0.3, skipping

In the example above n98-magerun is telling us it didn’t do anything, since the latest version of the Pulse Storm Launcher is already installed.  That points to one of the weaknesses of the extension category of commands.  While the commands here are useful, you’ll still need to rely on ./mage (or the Connect GUI) if you want to view a list of available upgrades, uninstall an extension, etc.
Wrap Up
And that completes our command coverage of n98-magerun.  There’s a few other commands that are worth checking out on your own.  The cms category has two commands that are Magento Enterprise Edition only, cms:banner:toggle for publishing a banner, and cms:page:publish for publishing a specific CMS page.  There’s also a design:demo-notice command which can toggle a demo notice on/off in your store.
Next time we’ll start digging into setting up your own n98-magerun development environment.
]]>
			http://mage-page.net/2013/04/n98-magerun-indexing-magento-connect-and-grab-bag/feed/
		0
		
		
		
		http://mage-page.net/2013/04/magento-community-edition-ce-1-8-0-0-alpha-now-available/
		http://mage-page.net/2013/04/magento-community-edition-ce-1-8-0-0-alpha-now-available/#comments
		Thu, 25 Apr 2013 04:10:00 +0000
		admin
				

		http://mage-page.net/2013/04/magento-community-edition-ce-1-8-0-0-alpha-now-available/
		We are excited to announce the release of Magento Community Edition 1.8.0.0- alpha1, available now for preview. This release focuses on major quality and stability improvements, performance enhancements and security


Key highlights include:

Performance Improvements

This release provides better performance in a number of ways, including improved performance of the checkout process, optimized cache adapters for single-server systems and speedier large database lookups. You can even load a large number of tax rates (3,000+) without any impact to performance.

Enhanced Tax Calculation Algorithms

This latest version of Magento Community Edition improves tax calculation algorithms, eliminating potential rounding offsets that can be displayed on buyer facing screens. This release also provides additional support for Canadian tax requirements.

Functional Improvements

We have made approximately 350 functional improvements in key ...]]>
				We are excited to announce the release of Magento Community Edition 1.8.0.0- alpha1, available now for preview. This release focuses on major quality and stability improvements, performance enhancements and security

Key highlights include:

Performance Improvements

This release provides better performance in a number of ways, including improved performance of the checkout process, optimized cache adapters for single-server systems and speedier large database lookups. You can even load a large number of tax rates (3,000+) without any impact to performance.

Enhanced Tax Calculation Algorithms

This latest version of Magento Community Edition improves tax calculation algorithms, eliminating potential rounding offsets that can be displayed on buyer facing screens. This release also provides additional support for Canadian tax requirements.

Functional Improvements

We have made approximately 350 functional improvements in key areas, including the web store, shopping cart, admin order creation, import and export functionality, web API components and payment methods.

Security Enhancements

Through a number of enhancements, we have also strengthened the security of Magento Community Edition. Check out the full list of enhancements in our release notes.
We encourage you to take this preview release for a test drive and experiment with it. Please note that we do not recommend that you use this release in a production environment. We will incorporate the feedback we get from the community along with any open issues, in an upcoming production-ready version. Learn more about alpha releases and the Magento Community Edition release process.