We've all been there, needing to migrate resources form one MODX website to another, this is a method we use from time to time. It works especially well when you have multiple template variables you need to match to new template variables.
Note: These principals can also be used for importing only if scraping data with other tools and building a custom import file.
MODX Extras Required
About importX
We love modmore here, but importX isn't exactly at the top of their list to improve and enhance, so we've had to work around some of the shortcomings of the CSV parsing in importX.
We first used CSV validators to try and get proper escaping of quotes or semicolons, but it never seemed to work right. Eventually after understanding that the CSV parsing is very rudimentary we realized "why not just use some crazy delimiter?" and it worked.
The delimiter or separator
<:>
After using the above string for our field delimiter, we've been able to import just about anything with no issues.
Custom Output Filters / Snippets
We've also come up with a few relatively simple output filters that help clean up the data so it imports properly. Create a new snippet and paste the code in.
Learn more about input / output filters
removeNonPrinatable
This removes hidden characters and many nonprintable strings. We use this mainly in the content field.
<?php
$output = preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $input);
return $output;
stripNL
Another small but useful tool, and possibly could be combined with the above output filters is the stripNL snippet. It simple strips newlines from fields. It works great on the content field, or any other rich text field. New lines will throw off the CSV parser of importX.
<?php $output=preg_replace( "/\r|\n/", "", $input ); return $output;
Exporting Data
Create a resource and hide it from menus, set template to empty and content type to text. You want a blank plain text file to output.
We're assuming some familiarity with pdoResources , there are many examples and guides on how to use and configure it.
Export Resource Content
Add any TVs you also need, or other fields you want like editedon or menutitle, make sure to update the template as well with the additional columns. Make sure to use the target sites TV ID as the column header. Example: tv34 - This will take the data in that column for each row and import into the target site's template variable with ID of 34. You can find the ID in the elements tree.
template<:>context_key<:>parent<:>published<:>publishedon<:>hidemenu<:>pagetitle<:>content<:>alias<:>isfolder<:>introtext<:>
[[!pdoResources?
&parents=`125`
&resources=``
&includeContent=`1`
&showUnpublished=`0`
&showHidden=`1`
&includeTVs=`` [[- Make sure to include all TV names you wish to import]]
&tpl=`export-tpl`
&limit=`0`
]]
pdoResources Export Template
Note the first 3 positions here are hard coded, we use this mainly for importing sections, rather than the site as a whole.
- template - Very important, set to the ID of the template that you'll be using in the NEW site.
- context_key - Very important for multi-context sites as web will be assumed.
- parent - Set to the parent ID, you can set this in importX, but it's good to be defensive and be sure.
When adding template variables make sure to add the appropriate column
14<:>web<:>330<:>[[+published]]<:>[[+publishedon]]<:>[[+hidemenu]]<:>[[+pagetitle]]<:>[[+content:stripNL:removeNonPrinatable]]<:>[[+alias]]<:>[[+isfolder]]<:>[[+introtext]]<:>
Exporting and Importing
First before doing a large run, do a small test to see if it works ok, then do the entire batch less the test data. You can use the pdoResources offset parameter for excluding a few rows.
Get the export data by visiting the resource you created for the pdoResources call.
Next copy that and paste it into importX, then make sure to add our custom delimiter or separator.
Last item to be aware of with importX is on subsequent runs you need to refresh the browser for each additional run and reset the delimiter or separator.
That's it, happy importing.