Bash Script: Create MySQL Database & User with Optional Password

January 3, 2017  |  No Comments  |  by Raj (MagePsycho)  |  Linux, Mysql

I have been using DigitalOcean for my personal projects. And one repeated things that I had to do is the creation of MySQL database & user. I know I could have used some CPanel like GUIs and even some of them came to my mind like VestaCP, Ajenti, ServerPilot etc. But all comes with sort of things and not so developer friendly.

So I decided to create a simple bash script that can simply the database & user.

Bash Script Code

You can find the script at Gist:

How to use?

1# Download
Download the script from Gist – Bash Script: Create MySQL Database & User

2# Set Permission
You need to set the executable permission in order to execute the shell script:

chmod +x mysql-create-db-user.sh

3# Usage
Use the script as:

./mysql-create-db-user.sh [--host="<host-name>"] --database="<db-name>" [--user="<db-user>"] [--pass="<user-password>"]

So the only required parameter is database name. In the case of empty values for other parameters:

  • ‘host’ becomes localhost
  • ‘user’ takes value from database name
  • ‘password’ is randomly generated

For the following command:

./mysql-create-db-user.sh --database=bash_db2

The output looks like:

Bash Script: Create MySQL Database & User with Optional Password

Bash Script Console: Create MySQL Database & User with Optional Password

Hope you find this script useful.
Please do let us know any feedback in the comments below.

Sandbox script for quick testing in Magento2

September 14, 2016  |  No Comments  |  by Raj (MagePsycho)  |  Linux, Mac OS-X, Magento 2

What’s your approach for quick & dirty testing in Magento2?

  • Creating a test module with a controller and executing it from browser to see the output?
  • Creating a Console module and executing via CLI?

Obviously, above two approaches takes time.

Rather I would create a simple script (a single file) and put it somewhere in /[path-to-magento2]/pub/sandbox.php

<?php
/**
 * @author Raj KB<magepsycho@gmail.com>
 * @website http://www.magepsycho.com
 */
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
require __DIR__ . '/../app/bootstrap.php';
$bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER);
$obj = $bootstrap->getObjectManager();
$state = $obj->get('Magento\Framework\App\State');
$state->setAreaCode('frontend');

// Your quick and dirty code goes here...
$quote = $obj->get('Magento\Checkout\Model\Session')->getQuote()->load(1);
Zend_Debug::dump($quote->getOrigData());

Now you can easily test by pointing to URL
http://[magento2-url]/sandbox.php

But wait, this won’t work in the case when you are using Nginx + PHP-FPM server.
The reason you can explore from the Nginx conf file: [path/to/magento2]/nginx.conf.sample

...
# PHP entry point for main application
location ~ (index|get|static|report|404|503)\.php$ {
    try_files $uri =404;
    fastcgi_pass   fastcgi_backend;
    fastcgi_buffers 1024 4k;

    fastcgi_param  PHP_FLAG  "session.auto_start=off \n suhosin.session.cryptua=off";
    fastcgi_param  PHP_VALUE "memory_limit=768M \n max_execution_time=600";
    fastcgi_read_timeout 600s;
    fastcgi_connect_timeout 600s;

    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    include        fastcgi_params;
}
...

As you can see only PHP files: index.php, get.php, static.php, report.php, 404.php & 503.php files are parsed by the PHP-FPM.

In order to include your sandbox.php in the whitelist, just edit the line of Nginx conf file:

location ~ (index|get|static|report|404|503)\.php$ {

to

location ~ (index|get|static|report|404|503|sandbox)\.php$ {

And don’t forget to reload or restart your Nginx server depending upon your Operating System.
Ubuntu

sudo service nginx reload
sudo service nginx restart

MacOSx

sudo nginx -s reload
sudo nginx -s stop && sudo nginx

What’s your approach for quick & dirty testing in Magento2? Please do comment below.

MySQL Issue: Table storage engine for ‘catalog_product_relation’ doesn’t have this option

July 30, 2016  |  1 Comments  |  by Raj (MagePsycho)  |  Linux, Mac OS-X, Magento, Mysql

When you export the Magento database dump from MySQL 5.5.x and try to import in MySQL 5.7.x, You are likely to face an error:

ERROR 1031 (HY000) at line 3002: Table storage engine for ‘catalog_product_relation’ doesn’t have this option

PROBLEM

This is probably due to the table option that you have in your CREATE TABLE DDL: ROW_FORMAT=FIXED

Let’s check if there is any such string in the SQL dump (Ex: magento-db-dump.sql).

cat magento-db-dump.sql | grep '=FIXED'

Which resulted as:

) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=FIXED COMMENT='Catalog Product Relation Table';
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=FIXED COMMENT='Catalog Product To Website Linkage Table';

Refer – MySQL Row Format Option

SOLUTION

Removing ROW_FORMAT=FIXED option from CREATE TABLE DDL will fix the issue.
So let’s try possible solutions.

#1

sed -i 's/ROW_FORMAT=FIXED//g' magento-db-dump.sql

This didn’t work for me in MacOSx which resulted in following error:

sed: 1: “magento-db-dump.sql”: invalid command code m

#2

sed -i '' 's/ROW_FORMAT=FIXED//g' magento-db-dump.sql

And even this resulted as:

sed: RE error: illegal byte sequence

#3
But this one worked for me in MacOSx (Refer Mac OS X, Sed, and strange document encoding to know more about the issue)

LC_ALL=C sed -i '' 's/ROW_FORMAT=FIXED//g' magento-db-dump.sql

Re-check if the string has been removed or not:

cat magento-db-dump.sql | grep '=FIXED'

If the string is removed, now try to import as:

mysql -u <user> -p <db-name> < magento-db-dump.sql

Yay! Now it imported successfully.
Please do share and care if you liked this article.

Cheers!

Install Bash Completion for Mac OS-X

May 22, 2016  |  No Comments  |  by Raj (MagePsycho)  |  Linux, Mac OS-X

One of the nicest features of the modern shell is the built in “completion” support. This allows you to complete commands and their arguments easily without memorizing.

Unfortunately, Mac OS-X bash shell doesn’t have completion feature by default unlike Ubuntu(Debian based Linux OS).
But the good thing is you can easily install the auto completion by using Homebrew or Macports.

Homebrew being my favourite packaging tool for OS-X, I will be explaining the installation using this tool.

Bash Completion Installation

1. Install bash-completion package using brew

brew install bash-completion

Brew Bash Completion

2. Edit the ~/.bash_profile

vi ~/.bash_profile

And add the following code:

if [ -f $(brew --prefix)/etc/bash_completion ]; then
    . $(brew --prefix)/etc/bash_completion
fi

3. Reload the bash shell

source ~/.bash_profile

Now try to type few letters of command and press [TAB], you will see the auto-completed command or auto-suggested commands(for more than one matches). For example:
nets[TAB] ->

netstat

net[TAB] ->

net-server               net-server5.18           net-snmp-config
net-server5.16           net-snmp-cert            net-snmp-create-v3-user

Bonus

You can also install additional completions from Homebrew-Completions
All you have to do is:

brew tap homebrew/completions
brew install <formula>

If you know how to create bash completions, you can create your own file and drop in the folder:

/usr/local/etc/bash_completion.d

After sourcing(loading) it, your custom completion is ready to take into account.

Please do share/comment on your favourite bash-completion.

Install & Configure Varnish Cache for Magento 2

May 19, 2016  |  13 Comments  |  by Raj (MagePsycho)  |  Linux, Magento 2

The performance of a website is always the key factor for the higher conversion rate. Fortunately, Magento 2 (both CE & EE) comes by default with the support for FPC (Full Page Caching), Varnish & Redis to make your store fly.
In this tutorial we will be discussing brief info about Varnish, it’s installation and configuration so bear with me.

Magento-2-Banner

Magento 2 supports Varnish versions 3.0.5 or later or any Varnish 4.x version.
And Magento team strongly recommends using Varnish in production as the built-in full-page caching (to either the file system or database) is much slower than Varnish, and Varnish is designed to accelerate HTTP traffic.
varnishcache_rgb-gimp2-alpha

Varnish Cache is an open source web application accelerator (also referred to as an HTTP accelerator or caching HTTP reverse proxy). Varnish stores (or caches) files or fragments of files in memory; this enables Varnish to reduce the response time and network bandwidth consumption on future, equivalent requests. Unlike web servers like Apache and Nginx, Varnish was designed for use exclusively with the HTTP protocol.

Assumption

We have carried out the Varnish installation in the following enviroment:

  • Ubuntu – 14.04 LTS
  • Nginx – 1.9.6
  • MySQL – 5.6.28
  • PHP – 5.6.20 (but PHP 7 is recommended)
  • Magento EE – 2.0.5

Installation

Run the following series of command as a root user (sudo su).

apt-get install apt-transport-https
curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -
echo "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1" \
     >> /etc/apt/sources.list.d/varnish-cache.list
apt-get update
apt-get install varnish

If everything goes right you should be able to see the Varnish version with:

varnishd -V

Configuration

Once Varnish is installed, you need to configure it and your web server(Nginx in our case).

Configure your Web Server(Nginx)

Configure your web server to listen on a port (say: 8080) other than the default port 80 because Varnish responds directly to incoming HTTP requests, not the web server.
For this add the ‘listen’ directive in ‘server’ block as:

server {
    listen 8080 default_server;
    server_name  your-mage2-store.dev;
    ...
}

to default nginx configuration file (generally located at /path/to/nginx/sites-enabled/default)

Configure Varnish Configuration

1. Modify Varnish System Configuration

vim /etc/default/varnish

And edit the file to have similar code:

START=yes

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Maximum locked memory size (for ulimit -l)
# Used for locking the shared memory log in memory.  If you increase log size,
# you need to increase this number as well
MEMLOCK=82000

DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -p http_resp_hdr_len=64000 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"

2. Modify default.vcl
Make a backup of default one.

cp /etc/varnish/default.vcl /etc/varnish/default.vcl.bak
vim /etc/varnish/default.vcl

And edit the lines under ‘backend default’:

# Default backend definition. Set this to point to your content server.
backend default {
.host = "127.0.0.1";
.port = "8080";
}

Where

  • .host – fully qualified host name or IP address and listen port of the Varnish backend or origin server; that is, the server providing the content Varnish will accelerate. Typically, this is your web server.
  • .port – the web server’s listen port (8080 in this example)

Note:This is the minimal configuration. You need to use the configuration which you get from Magento 2’s Varnish Export (Ref – #3).

3. Configure Magento to use Varnish
Instead of manually editing default.vcl, Magneto 2 comes up with an export option for it.
First enable the Full Page Cache to use Varnish (Go to Admin > Stores > Configuration > Advanced > System > Full Page Cache), you will see as:
magepsycho-magento-2-varnish-cache
Save the settings and hit the Export VCL button (Varnish 3 or 4 as per your Varnish Version).
And use this default.vcl to replace the file in /etc/varnish/default.vcl

Finally, restart the Varnish & Nginx

service nginx restart
service varnish restart

Verification

I assume You have followed all the above steps (Installation & Configuration). Now lets check if Varnish cache server is taking into account for HTTP response or not.

Check if Varnish is listening to port 80
Run the following command:

netstat -tulpn | grep varnishd

varnishd-listing-toport-80

Verify contents in var/cache, var/page_cache folder is not regenerated
Since FPC is configured to serve the content from Varnish, once cleared, cache folders (var/cache, var/page_cache) should always be empty .

Verify the HTTP response headers
Load your Magento 2 Frontend and check the response from browser console, you will see similar as shown in the below screenshot:
magepsycho-varnish-http-response-headers

Or you can simply use CLI as:

curl -I http://your-mage2-store.dev/

And You will see some similar Varnish keywords.

Voila, you’re done. Let your store fly!
Please do comment or contact us if you have any issues regarding Varnish + Magento2 Installation & Configuration.