Migrator class - usage and performance questions


I’m using the Migrator class together with a bunch of macros and templates to import json files into Grakn.

Everything goes well, until it doesn’t :slight_smile:

  1. It seems, that when something goes wrong inserting a single entity/relation, the whole batch task goes to failed status (I can see that in the dashboard). I can get why, but I can’t seem to find anywhere what has gone wrong in the logs of the engine. I’ve been able to figure it out by using the GraphBuilder.parseTemplate and sending individual json entries, but obviously this is time consuming and not something I want to do each time something goes wrong with the input data.

  2. Which mode does the Migrator use? Does it use batch? I’m just asking because I’m getting validation errors – that’s nice, but I thought these were disabled during batch processing.
    Importing relationships can go rather slow with at least 2 matches and an insert per relation.


Hi @kccoyote,

With regards to the failure not being logged or visible enough during migration this is a known issue for us and we busy working on it.

Which mode does the Migrator use? Does it use batch?

It uses batch mode but batch mode does not disable validation. At this stage there is no way to disable validation because if a validation error were ignored or missed the underlaying object model would become partially corrupted and there is no simple way to recover from this. In the future we may add an option to disable validation but its not something we looking to do for the moment.

Maybe if you share the errors you are seeing we may be able to help you work around your current problems.




Thanks for the info.

As mentioned, as soon as I knew what the error was, I was able to rectify it. The problem was more so that it took me more time to figure out why the migration failed than it took me to fix the error.

It’s good to know this is being worked on.