Recommendation to import data


I am using mostly python and tried to test import a larger dataset into grakn.

The development of a schema worked very well and I am really impressed with the expressiveness of the system.
Yet I am still facing some problems importing data which I translated from an internal data source.

My first attempt was to use the REST API – reading worked fine but writing was a problem as the return error was always that my query is “read-only” – is it possible to use the API for writing with a configuration change or is this planned for a coming version?

My second attempt was to write files which I can import into grakn with the graql shell. Using the –f and –b flag resulted in out of memory errors at first so I tried using a instance-id resource for every entity to be used for relations like in the movies sample dataset.

The problem I encountered was that it looks like using the with the –b flags gives me different graphs with some relations missing which I can see if I import with –f and which are present in the file with the insert statements (I am using version 0.12.1).

Do you have a recommendation for me which way to use to import data?



With regards to the REST API, we have added in a POST endpoint for executing insert queries, but I think that was after the 0.12.1 release. It will be available in the next release.

The batch loader (-b) is multithreaded and we cannot guarantee that the queries are executed in the order they appear in the file. I suspect that the file you were trying to load contains both relations and entities. It is possible that some of the match-insert relation queries were executed before the entities those relations are between were inserted. This means that, yes, some relations just wouldn’t show up in the final graph.

We definitely recommend that you continue to use the -b flag when loading larger quantities of data (-f being single threaded). Your best bet would be to move the relations and entities into separate files and just make sure to load the entities first.

Im not sure about the memory errors - will make an internal note to look into that.

Hope this helps!


I will be able to split the import into entities and relations without much effort - this sounds like a good solution!

With the API I would be able to get rid of the “secondary” id I am using right know and use the system generated id - I am looking forward to the next release.

Many Thanks!


Just wanted to confirm that the solution you suggested works flawlessly and fast !


I would like to take advantage of the mention of the new REST API POST endpoint, and ask about the next release tentative date, I’m pretty interested in this due to I’m planning to connect a .Net Core API to GRAKN via the REST API, thanks.


That’s very exciting!

We are going to aim to release in the next week or two. If you are eager to get started, check out the stable branch on github–– this, plus incoming bug fixes, is the code that we will release for 0.13.0.


Great news, thanks!

Yesterday I made my way around by (temporary) calling the shell from Net Core and running my insertion scripts.

In the meanwhile, I’ll be reading and absorbing your docs.