MySQL Issue: Table storage engine for ‘catalog_product_relation’ doesn’t have this option

July 30, 2016  |  1 Comments  |  by Raj (MagePsycho)  |  Linux, Mac OS-X, Magento, Mysql

When you export the Magento database dump from MySQL 5.5.x and try to import in MySQL 5.7.x, You are likely to face an error:

ERROR 1031 (HY000) at line 3002: Table storage engine for ‘catalog_product_relation’ doesn’t have this option

PROBLEM

This is probably due to the table option that you have in your CREATE TABLE DDL: ROW_FORMAT=FIXED

Let’s check if there is any such string in the SQL dump (Ex: magento-db-dump.sql).

cat magento-db-dump.sql | grep '=FIXED'

Which resulted as:

) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=FIXED COMMENT='Catalog Product Relation Table';
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=FIXED COMMENT='Catalog Product To Website Linkage Table';

Refer – MySQL Row Format Option

SOLUTION

Removing ROW_FORMAT=FIXED option from CREATE TABLE DDL will fix the issue.
So let’s try possible solutions.

#1

sed -i 's/ROW_FORMAT=FIXED//g' magento-db-dump.sql

This didn’t work for me in MacOSx which resulted in following error:

sed: 1: “magento-db-dump.sql”: invalid command code m

#2

sed -i '' 's/ROW_FORMAT=FIXED//g' magento-db-dump.sql

And even this resulted as:

sed: RE error: illegal byte sequence

#3
But this one worked for me in MacOSx (Refer Mac OS X, Sed, and strange document encoding to know more about the issue)

LC_ALL=C sed -i '' 's/ROW_FORMAT=FIXED//g' magento-db-dump.sql

Re-check if the string has been removed or not:

cat magento-db-dump.sql | grep '=FIXED'

If the string is removed, now try to import as:

mysql -u <user> -p <db-name> < magento-db-dump.sql

Yay! Now it imported successfully.
Please do share and care if you liked this article.

Cheers!

How to fix the issue: Product images missing in backend but not in frontend

June 22, 2012  |  50 Comments  |  by Raj (MagePsycho)  |  Latest, Magento

Introduction

Recently i have to fix a strange issue for one of my client. Issue was that product images were missing from ‘Images’ tab of product edit form (as depicted below), though they were displaying fine in the frontend.

Missing Product Images In Backend

After going through the table relationship for catalog product images (as shown below), it was found that required rows were missing in table ‘catalog_product_entity_media_gallery’. Note that table ‘catalog_product_entity_media_gallery_value’ is not necessary unless you want to sort or label the product images.

Schema For Product Images

By inspecting the html code of Images tab of product edit form via firebug, it was seen that ‘value’ attribute had empty json:

<input type="hidden" value="[]" name="product[media_gallery][images]" id="media_gallery_content_save">

Further Debugging:
Using template path hints extension for admin: Easy Template Path Hints, it was found that the responsible template and block class were: ‘app/design/adminhtml/default/default/template/catalog/product/helper/gallery.phtml’ and ‘Mage_Adminhtml_Block_Catalog_Product_Helper_Form_Gallery_Content’ respectively.

Looking at the block class and .phtml code it was found that Input element has empty json because of Mage_Adminhtml_Block_Catalog_Product_Helper_Form_Gallery_Content::getImagesJson() method.

When populated the images from table ‘catalog_product_entity_varchar’ to ‘catalog_product_entity_media_gallery’ for those products, products images were seen at the backend(refer to the code below for more info).
Here goes the steps for fixing:

Steps

1> Prepare CSV file(update_missing_images.csv) with only one field: sku and upload to the root of Magento installation.
Note: This will contain the sku of those products whose images are missing at the backend.

2> Create a file: update_missing_images.php and upload to the root of magento installation and paste the following code:

<?php
/**
 * @author		MagePsycho <info@magepsycho.com>
 * @website		http://www.magepsycho.com
 * @category	Export / Import
 */
$mageFilename = 'app/Mage.php';
require_once $mageFilename;
Mage::setIsDeveloperMode(true);
ini_set('display_errors', 1);
umask(0);
Mage::app('admin');
Mage::register('isSecureArea', 1);
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

set_time_limit(0);
ini_set('memory_limit','1024M');

/***************** UTILITY FUNCTIONS ********************/
function _log($message, $file = 'update_missing_images.log'){
	Mage::log($message, null, $file);
}

function _getIndex($field) {
	global $fields;
	$result = array_search($field, $fields);
	if($result === false){
		$result = -1;
	}
	return $result;
}

function _getConnection($type = 'core_read'){
	return Mage::getSingleton('core/resource')->getConnection($type);
}

function _getTableName($tableName){
	return Mage::getSingleton('core/resource')->getTableName($tableName);
}

function _getAttributeId($attribute_code = 'price'){
	$connection = _getConnection('core_read');
	$sql = "SELECT attribute_id
				FROM " . _getTableName('eav_attribute') . "
			WHERE
				entity_type_id = ?
				AND attribute_code = ?";
	$entity_type_id = _getEntityTypeId();
	return $connection->fetchOne($sql, array($entity_type_id, $attribute_code));
}

function _getEntityTypeId($entity_type_code = 'catalog_product'){
	$connection = _getConnection('core_read');
	$sql		= "SELECT entity_type_id FROM " . _getTableName('eav_entity_type') . " WHERE entity_type_code = ?";
	return $connection->fetchOne($sql, array($entity_type_code));
}

function _getIdFromSku($sku){
	$connection = _getConnection('core_read');
	$sql		= "SELECT entity_id FROM " . _getTableName('catalog_product_entity') . " WHERE sku = ?";
	return $connection->fetchOne($sql, array($sku));
}

function _checkIfSkuExists($sku){
	$connection = _getConnection('core_read');
	$sql		= "SELECT COUNT(*) AS count_no FROM " . _getTableName('catalog_product_entity') . "	WHERE sku = ?";
	$count		= $connection->fetchOne($sql, array($sku));
	if($count > 0){
		return true;
	}else{
		return false;
	}
}

function _checkIfRowExists($productId, $attributeId, $value){
	$tableName  = _getTableName('catalog_product_entity_media_gallery');
	$connection = _getConnection('core_read');
	$sql		= "SELECT COUNT(*) AS count_no FROM " . _getTableName($tableName) . " WHERE entity_id = ? AND attribute_id = ?  AND value = ?";
	$count		= $connection->fetchOne($sql, array($productId, $attributeId, $value));
	if($count > 0){
		return true;
	}else{
		return false;
	}
}

function _insertRow($productId, $attributeId, $value){
	$connection				= _getConnection('core_write');
	$tableName				= _getTableName('catalog_product_entity_media_gallery');

	$sql = "INSERT INTO " . $tableName . " (attribute_id, entity_id, value) VALUES (?, ?, ?)";
	$connection->query($sql, array($attributeId, $productId, $value));
}

function _updateMissingImages($count, $productId, $data){
	$connection				= _getConnection('core_read');
	$smallImageId			= _getAttributeId('small_image');
	$imageId				= _getAttributeId('image');
	$thumbnailId			= _getAttributeId('thumbnail');
	$mediaGalleryId			= _getAttributeId('media_gallery');

	//getting small, base, thumbnail images from catalog_product_entity_varchar for a product
	$sql	= "SELECT * FROM " . _getTableName('catalog_product_entity_varchar') . " WHERE attribute_id IN (?, ?, ?) AND entity_id = ? AND `value` != 'no_selection'";
	$rows	= $connection->fetchAll($sql, array($imageId, $smallImageId, $thumbnailId, $productId));
	if(!empty($rows)){
		foreach($rows as $_image){
			//check if that images exist in catalog_product_entity_media_gallery table or not
			if(!_checkIfRowExists($productId, $mediaGalleryId, $_image['value'])){
				//insert that image in catalog_product_entity_media_gallery if it doesn't exist
				_insertRow($productId, $mediaGalleryId, $_image['value']);
				/* Output / Logs */
				$missingImageUpdates = $count . '> Updated:: $productId=' . $productId . ', $image=' . $_image['value'];
				echo $missingImageUpdates.'<br />';
				_log($missingImageUpdates);
			}
		}
		$separator = str_repeat('=', 100);
		_log($separator);
		echo $separator . '<br />';
	}
}
/***************** UTILITY FUNCTIONS ********************/

$messages			= array();
$csv				= new Varien_File_Csv();
$data				= $csv->getData('update_missing_images.csv'); //path to csv
$fields				= array_shift($data);
#print_r($fields); print_r($data); exit;

$message = '<hr />';
$count   = 1;
foreach($data as $_data){
	$sku									= isset($_data[_getIndex('sku')]) ? trim($_data[_getIndex('sku')]) : '';
	if(_checkIfSkuExists($sku)){
		try{
			$productId = _getIdFromSku($sku);
			_updateMissingImages($count, $productId, $_data);
			$message .= $count . '> Success:: While Updating Images of Sku (' . $sku . '). <br />';

		}catch(Exception $e){
			$message .=  $count .'> Error:: While Upating Images of Sku (' . $sku . ') => '.$e->getMessage().'<br />';
		}
	}else{
		$message .=  $count .'> Error:: Product with Sku (' . $sku . ') does\'t exist.<br />';
	}
	$count++;
}
echo $message;

3> Open your browser and run the following url:
http://your-magento-url/update_missing_images.php

4> That’s all. Try to browse the Images tab, now you will see those missing images.

Hope this helps somebody!

Updating product prices in Magento in an easier & faster way

November 4, 2011  |  67 Comments  |  by Raj (MagePsycho)  |  Latest, Magento, Mysql

Introduction

Product prices can be updated via default import profile though but this is very slow and requires lots of csv fields(besides sku & price) for updating.
Today i am going to talk about updating product prices just by using csv with two fields: sku & price(new) which is very fast enough even for thousands of products.

Steps:

1> Prepare CSV file(prices.csv) with two fields: sku & price and upload in the root of Magento installation.
You can see the snapshot how it should look like:

prices.csv

prices.csv


Note: Field/Values should be enclosed within double quotes(“) and separated by a comma(,). I would prefer OpenOffice for csv formatting.

2> Create a file: update_prices.php in the root of magento installation and paste the following code:

<?php
/**
 * @author		MagePsycho <info@magepsycho.com>
 * @website		http://www.magepsycho.com
 * @category	Export / Import
 */
$mageFilename = 'app/Mage.php';
require_once $mageFilename;
Mage::setIsDeveloperMode(true);
ini_set('display_errors', 1);
umask(0);
Mage::app('admin');
Mage::register('isSecureArea', 1);
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

set_time_limit(0);
ini_set('memory_limit','1024M');

/***************** UTILITY FUNCTIONS ********************/
function _getConnection($type = 'core_read'){
	return Mage::getSingleton('core/resource')->getConnection($type);
}

function _getTableName($tableName){
	return Mage::getSingleton('core/resource')->getTableName($tableName);
}

function _getAttributeId($attribute_code = 'price'){
	$connection = _getConnection('core_read');
	$sql = "SELECT attribute_id
				FROM " . _getTableName('eav_attribute') . "
			WHERE
				entity_type_id = ?
				AND attribute_code = ?";
	$entity_type_id = _getEntityTypeId();
	return $connection->fetchOne($sql, array($entity_type_id, $attribute_code));
}

function _getEntityTypeId($entity_type_code = 'catalog_product'){
	$connection = _getConnection('core_read');
	$sql		= "SELECT entity_type_id FROM " . _getTableName('eav_entity_type') . " WHERE entity_type_code = ?";
	return $connection->fetchOne($sql, array($entity_type_code));
}

function _getIdFromSku($sku){
	$connection = _getConnection('core_read');
	$sql		= "SELECT entity_id FROM " . _getTableName('catalog_product_entity') . " WHERE sku = ?";
	return $connection->fetchOne($sql, array($sku));

}

function _checkIfSkuExists($sku){
	$connection = _getConnection('core_read');
	$sql		= "SELECT COUNT(*) AS count_no FROM " . _getTableName('catalog_product_entity') . "	WHERE sku = ?";
	$count		= $connection->fetchOne($sql, array($sku));
	if($count > 0){
		return true;
	}else{
		return false;
	}
}

function _updatePrices($data){
	$connection		= _getConnection('core_write');
	$sku			= $data[0];
	$newPrice		= $data[1];
	$productId		= _getIdFromSku($sku);
	$attributeId	= _getAttributeId();

	$sql = "UPDATE " . _getTableName('catalog_product_entity_decimal') . " cped
				SET  cped.value = ?
			WHERE  cped.attribute_id = ?
			AND cped.entity_id = ?";
	$connection->query($sql, array($newPrice, $attributeId, $productId));
}
/***************** UTILITY FUNCTIONS ********************/

$csv				= new Varien_File_Csv();
$data				= $csv->getData('prices.csv'); //path to csv
array_shift($data);

$message = '';
$count   = 1;
foreach($data as $_data){
	if(_checkIfSkuExists($_data[0])){
		try{
			_updatePrices($_data);
			$message .= $count . '> Success:: While Updating Price (' . $_data[1] . ') of Sku (' . $_data[0] . '). <br />';

		}catch(Exception $e){
			$message .=  $count .'> Error:: While Upating  Price (' . $_data[1] . ') of Sku (' . $_data[0] . ') => '.$e->getMessage().'<br />';
		}
	}else{
		$message .=  $count .'> Error:: Product with Sku (' . $_data[0] . ') does\'t exist.<br />';
	}
	$count++;
}
echo $message;

3> Open your browser and run the following url:
http://your-magento-url/update_prices.php
You will see how fast the prices of corresponding skus are updated.
I have tested with 2000 proudcts and update was accomplished within seconds. Isn’t that fast enough?

EDIT
Recently we have developed an extension for importing all types of prices(regular, special, tier & group prices) via web-interface using the same raw SQL concept. Here goes the link for the extension:
Mass Importer Pro: Price Importer (Regular, Sepcial, Tier & Group Prices)

Thanks for reading. Please share your success story with this code.

Happy Importing!!

How to import product images from external url in magento

October 27, 2011  |  23 Comments  |  by Raj (MagePsycho)  |  Magento, PHP

Introduction

Generally, we import the product images by copying all the images to /media/import folder, adding image path relative to /media/import in csv corresponding to the product skus and running the import profile.

Sometimes you may need to import images from external url and this is the case when you are using third party web services or migrating non magento cart to magento.

Steps

Suppose say we have a csv with an extra field: external_image_url for product import.

1> Copy app/code/core/Mage/Catalog/Model/Convert/Adapter/Product.php to the local code pool so that new path looks like:
app/code/local/Mage/Catalog/Model/Convert/Adapter/Product.php

2> Open the Product.php from local code pool and search for the following code within saveRow() function:

foreach ($imageData as $file => $fields) {
	try {
		$product->addImageToMediaGallery(Mage::getBaseDir('media') . DS . 'import' . trim($file), $fields);
	}
	catch (Exception $e) {}
}

Add the following code just below it:

/** EXTERNAL IMAGE IMPORT - START **/
$image_url  = $importData['external_image_url']; //get external image url from csv
$image_type = substr(strrchr($image_url,"."),1); //find the image extension
$filename   = md5($image_url . $importData['sku']).'.'.$image_type; //give a new name, you can modify as per your requirement
$filepath   = Mage::getBaseDir('media') . DS . 'import'. DS . $filename; //path for temp storage folder: ./media/import/
file_put_contents($filepath, file_get_contents(trim($image_url))); //store the image from external url to the temp storage folder
$mediaAttribute = array (
		'thumbnail',
		'small_image',
		'image'
);
/**
 * Add image to media gallery
 *
 * @param string        $file              file path of image in file system
 * @param string|array  $mediaAttribute    code of attribute with type 'media_image',
 *                                         leave blank if image should be only in gallery
 * @param boolean       $move              if true, it will move source file
 * @param boolean       $exclude           mark image as disabled in product page view
 */
$product->addImageToMediaGallery($filepath, $mediaAttribute, false, false);
/** EXTERNAL IMAGE IMPORT - END **/

3> That’s all. Now you can proceed with normal dataflow import profile.

From above code it is clear that we have used the php functions: file_get_contents() & file_put_conents() for the purpose.
Be sure to check if the fopen wrappers have been enabled or not in order to use url as parameter for file_get_contents(). Refer the configuration directive: allow-url-fopen for more info.

Let me know your feedback regarding this article.

Thanks for reading.

EDIT: FYI, this code was tested in Magento 1.5.x version.

Export / Import CMS pages / Static Blocks Via SSH

October 10, 2011  |  4 Comments  |  by Raj (MagePsycho)  |  Linux, Magento, Mysql

It’s really a tedious task when you have to copy all the cms pages / static blocks from your development server to the live server. If you are importing the whole database then you are lucky enough but if you are updating in already installed magento server then :(, luckily you can use the following commands via SSH:

1> Export from your development server:

mysqldump -u [username] -p[password] [database] cms_page cms_page_store cms_block cms_block_store | grep INSERT | sed 's/INSERT INTO/REPLACE INTO/' > cms-export.sql

Note: Don’t forget to add prefix to cms_* tables if you have prefixed your tables.

2> Import to your live/destination server:

mysql -u [username] -p[password] [database] < cms-export.sql

Hope this saves your time while copying cms pages & static blocks.

Let’s share if anyone has better idea.

Happy E-Commerce!!