hey there,

here it comes, my first really helpfull post, hopefully. So for my current project we are dealing with more than 20.000 items, and all these items are linked to each other to provide a really huge and rich application. You can see the current Site here - think about importing the whole content.

I came here a couple of weeks after they have started the project and imported ~5.000 items into the sitecore tree already. So that was pretty fast and not a real problem. They used the XmlImporter free module out of the Sitecore SDN and we have applied a couple of improvements, like checking if the item already exists that you are not having 3 items of the same content, running the importer from a webservice to run the import out of an windows application etc.

After importing a lot more content the importer takes about 30 seconds per item title (and subitems and cross links). Well for around >20.000 items this cannot be the solution. 

So we have profiled the import of one item and found a couple of issues. After downloading the “original” XmlImporter I have seen that the lines I have modified not exist in the original one. But I can give you a couple of advices:

Avoid this:

Item item = parent.Items[”some key”];

to get the item, if you are dealing with that count of items, it takes a while to search the tree for the key.

Use this instead:

Item item = Sitecore.Configuration.Factory.GetDatabase(”master”).SelectSingleItem(string.Format(”{0}/{1}”,parent.Paths.Fullpath, “some key”));

To be honest, I’m not sure that this is the perfect solution be we have improved the speed by our importer from 30 sec / item title to ~2 sec / item title, depends of the cross links the item has.

Update: In order to Lars F. Nielsen, who remind me of the fact that the I did not mentioned that we are using our workstations and a shared database in our development enviroment. And also I made a mistake by saying that it is an item which takes about ~3 seconds. It is the title which takes about 3 seconds, and that includes a couple of Items which are created as well. See the comments for more information.