Opentaps Magento Integration

From Opentaps Wiki
Revision as of 22:58, 19 April 2010 by Oandreyev (talk | contribs) (Introduction)
Jump to navigationJump to search

This module is now ready for beta testing by developers. If you are interested, please contact Open Source Strategies to request access to the Subversion repository.


This integration allows you to use the Magento e-commerce system with opentaps Open Source ERP + CRM. The integration is designed to allow you to set up your products and catalogs in Magento, take orders using the Magento online store or using the Magento order entry module, and then import the products and orders to opentaps for purchasing, manufacturing fulfillment, and invoicing.

Specific features supported are:

  • One way push from Magento of newly added products and sales orders (Changes to products and orders are currently not handled.)
  • All Magento orders added to one "Magento Customer" in opentaps
  • Payments received in Magento are imported into opentaps under "Magento customer" an associated with the corresponding order
  • Once orders are shipped in opentaps, the tracking number is sent back to Magento and associate it with the original order.
  • Only one magento product store is supported at any time.

This integration is compatible with the free Magento Community Edition, tested with version

How It Works

An opentaps integration module for Magento retrieves product, customer, and order information and serves it via REST to opentaps. There is an opentaps Data Access Layer (dal) which uses the Pentaho Data Integration (kettle) to map the data from Magento and store them into the staging tables for opentaps Data Import module. From there, the opentaps Data Import module imports the data into opentaps.


The opentaps-Magento integration requires one module for Magento and two components for opentaps:

  • IntegratingWeb module for Magento
  • Data Access Layer (DAL) and magento-integration components for opentaps

The Magento module retrieves product and order data from Magento and serves them up via the REST protocol. The DAL component for opentaps receives the data from the Magento module and stores them in the opentaps database, and the magento-integration component uses the opentaps data import component to import the products and orders.

Getting the Modules

Download the OpentapsIntegration module and the MagentoIntegration module from Sourceforge opentaps Files under the "Modules" directory. The OpentapsIntegration zip file contains a module which will be installed on your Magento instance, and the MagentoIntegration zip file contains two modules for opentaps.

Getting the Source Code from Subversion

You can also install the source code version from subversion by checking out the IntegratingWeb module for Magento and the Data Access Layer (dal) and magento integration modules for opentaps.

You would need to checkout the OpentapsIntegration module for Magento in $magentoInstallationPath/app/code/local

 $ svn co svn:// OpentapsIntegration 

Then, checkout the Data Access Layer (DAL) and magento-integration modules for opentaps from the hot-deploy sub-directory of opentaps:

 $ svn co svn:// dal  
 $ svn co svn:// magento-integration 

Installing the Magento Extension

1. If you downloaded the modules, then first unzip the OpentapsIntegration zip file and move the OpentapsIntegration directory and all its subdirectories to the /app/code/local/ directory of your Magento instance.

2. Make sure that your Magento directory structure must look like the following:


Move $magentoInstallationPath/app/code/local/OpentapsIntegration/Restful/etc/OpentapsIntegration_Restful.xml in $magentoInstallationPath/app/etc/modules:

  mv $magentoInstallationPath/app/code/local/OpentapsIntegration/Restful/etc/OpentapsIntegration_Restful.xml $magentoInstallationPath/app/etc/modules

3. Go on the Magento backend in System->Configuration, then Advanced->Advanced (at the bottom of the left-hand side tabs for the System->Configuration screen.)

4. Click on "SAVE CONFIG" so that the Magento configuration files will be reloaded.

5. Now click on "Disable modules output", make sure you see a module named OpentapsIntegration_Restful. If that is your case, then you have succesfully installed the Magento module:

Magento Modules Opentaps Integration.png

Installing the opentaps Modules

If you downloaded the MagentoIntegration modules for opentaps, then unzip the contents into the hot-deploy/ subdirectory of opentaps.

Your opentaps directory structure must look like the following:


Modify the file $opentapsInstallationPath/hot-deploy/component-load.xml uncomment the following two lines:

  <load-component component-location="dal"/>
  <load-component component-location="magento-integration"/>

Go back to the opentaps directory and rebuild everything:

opentaps $ ./ant 

The opentaps modules are now installed. Now restart opentaps, and go to http://localhost:8080/magento-integration/control/main to ensure that the modules are loaded.

Finally, load the seed data for Magento integration by using Web Tools > XML Import and loading the files hot-deploy/magento/data/MagentoSeedData.xml

Configuring the opentaps Magento Integration

First you need a user which has access to magento data. You can create this user from the Magento backend:

  1. In the Magento backend, go to System-> Web Service -> Users, and add a new user.
  2. Then go to System-> Web Service -> Roles and create a new Role with "Resource Access" set to "All".
  3. Finally, associate the user with that role.

Next, $opentapsInstallationPath/hot-deploy/magento-integration/config/

  1. Set the URL and administrative access user and password for opentaps
  2. Set the base store URL and resource access user and password for magento. Test it by trying the URL plus /ws/orders/login For example, if your base store URL is, then you should see a page at The password is also the same as your magento user's API key.
  3. Set paging size, which is the number of products retrieved in each batch from magento
  4. Set the magento store ID, which is the field store_id of the table core_store of the Magento DB.

NOTE: when you modify any .properties file in Opentaps, you must reboot your instance because properties are loaded statically at boot application time.

IMPORTANT NOTE: make sure that the system time in both Magento and opentaps are the same, so that the dates on the invoices and payments in opentaps are correct.

Opentaps/Magento Mappings explanation

The magento-integration Opentaps module has a directory named "mappings" that contains a CSV file for every Magento workflow/area currently supported by the integration. These CSV files are processed by the transformations, and used when storing data in Opentaps.

storeMappings.csv: handles the mapping between the Magento stores IDs and the Opentaps ProductStore/RootCategory entities.


The ProductCategory and ProductStore definitions can be found in magento-integration/data: by default XML files there are loaded as demo data. Informations about the Magento storeIDs can be found in the core_store table.

shippingMappings.csv: handles the mapping between the Magento shipping methods and the Opentaps shipping methods


The Opentaps shipping method IDs are defined as seed data, and are common of every Opentaps installation. The Magento shipping methods can be found in the core_config_data table: by default Magento append the carrier code (ex.: UPS) to the shipping method (ex.: GND, states for Ground), thus resulting in a shippingMethodId of ups_GND.

paymentsMappings.csv: handles the mapping between Magento payment gateways and Opentaps payment methods.


The Opentaps payment method types and be found in the PaymentMethodType entity. The Magento payment methods can be found in the core_config_data table.

taxIdMappings.csv: handles the mapping between Magento tax codes and Opentaps tax authorities.

  US-CA-*-Rate 1,CA_BOE
  US-NY-*-Rate 1,NY_DTF

The Opentaps tax authorities can be found in the TaxAuthority entity (taxAuthPartyId field). The Magento tax codes, that needs to be defined for tax calculation, can be found in the tax_calculation_rate table.

Testing the Integration

Magento orders need to be in the "Processing" state to be imported into opentaps. This means that they must have payment received first. You can use the paypal developer sandbox to set up a dummy paypal account for payment processing. Alternatively, you can create your order with Check/Money Order and invoice the order, and then it will be "Processing."

NOTE: The Payment Action of the Paypal method that will be used must be configured to SALE.

How-To Implement Paypal Payments Pro into Magento - SANDBOX

You can follow the [1] to set up a developer account for payment processing.

Performance tuning

When importing a large number of products/orders (order of thousands, we made some tests with more than 10.000 products) performance issues become important.


Default values of Opentaps 1.4 defined in work fine. Ensure that the Java Virtual Machine will be started with the following values

  -Xms512M -Xmx1536M -XX:MaxPermSize=1024M

Note: in Unix systems the maximum number of open file descriptors is sometimes configured for a non-production system, that probably will not have high loads. In Linux based systems, this value is most of the time configured to 1024: the value can be verified with the following command:

  ulimit -n

To increase the default value, add a line such as the following in the file /etc/security/limits.conf

  yourUser - nofile 10000 

Finally, reboot your machine.


In $magentoInstallationPath/.htaccess memory used by PHP must be increased to a value of at least 512 Mb: this is a value you must tune based on the number of concurrent users that will access you production web site.

  php_value memory_limit 512M