Set db directory as a docker volume cause grakn server fail to start


#1

Graknversion - 1.3.0
**Your operating system
** - CentOS 7
**Whether it is Grakn distribution that you download, installed from brew, or Docker
** - from Grakn GitHub release page
####The steps which you did
Step 1, I write the following dockerfile in order to build my grakn image

FROM openjdk:8-jdk

ARG GRAKN_VERSION=1.3.0
ENV GRAKN_HOME=/opt/grakn

RUN mkdir -p $GRAKN_HOME && \
    curl -sSL https://github.com/graknlabs/grakn/releases/download/v${GRAKN_VERSION}/grakn-dist-${GRAKN_VERSION}.tar.gz | \
    tar -C $GRAKN_HOME --strip-components=1 -xzv

ENV PATH=$PATH:$GRAKN_HOME
WORKDIR $GRAKN_HOME

COPY docker-entrypoint /usr/local/bin
RUN chmod a+x /usr/local/bin/docker-entrypoint

# Grakn Server
EXPOSE 4567
# Thrift client API
EXPOSE 9160
# Grakn gRPC
EXPOSE 48555

ENTRYPOINT [ "docker-entrypoint" ]

and the docker-entrypoint file looks like below

#!/bin/sh

grakn server start
tail -f -n +1 $GRAKN_HOME/logs/grakn.log

Step 2, I run the following docker cmmand to build the image and run a container

docker build -t grakn .
docker run -dt -v $(pwd)/db/:/opt/grakn/db/ -p 4567:4567 -p 9160:9160 -p 48555:48555 --name grakn grakn

Step 3, I check the container and found that grakn server failed to start, while I type grakn server status I see the following results

Storage: RUNNING
Queue: NOT RUNNING
Engine: NOT RUNNING

The current state of Grakn, by running these commands:

**ls -la /tmp/*.pid
**
-rw-r--r--. 1 root root 3 Aug 18 14:41 /tmp/grakn-storage.pid
**ps -ef | grep redis-server
**
root 489 476 0 15:20 pts/1 00:00:00 grep redis-server
jps (if this command is installed on the system. usually is, when you are on Oracle JDK)

113 CassandraDaemon
491 Jps

The logs. If they’re quite long, maybe try only posting the last 200 lines of each

- grakn.log
2018-08-18 15:12:49,332 [main] INFO  ai.grakn.engine.GraknConfig - Project directory in use: /opt/grakn
2018-08-18 15:12:49,335 [main] INFO  ai.grakn.engine.GraknConfig - Configuration file in use: /opt/grakn/conf/grakn.properties
2018-08-18 15:12:49,366 [main] INFO  ai.grakn.engine.GraknConfig - Project directory in use: /opt/grakn
2018-08-18 15:12:49,366 [main] INFO  ai.grakn.engine.GraknConfig - Configuration file in use: /opt/grakn/conf/grakn.properties
2018-08-18 15:15:47,497 [main] INFO  ai.grakn.engine.GraknConfig - Project directory in use: /opt/grakn
2018-08-18 15:15:47,500 [main] INFO  ai.grakn.engine.GraknConfig - Configuration file in use: /opt/grakn/conf/grakn.properties
2018-08-18 15:15:47,529 [main] INFO  ai.grakn.engine.GraknConfig - Project directory in use: /opt/grakn
2018-08-18 15:15:47,529 [main] INFO  ai.grakn.engine.GraknConfig - Configuration file in use: /opt/grakn/conf/grakn.properties
2018-08-18 15:15:47,530 [main] INFO  ai.grakn.engine.GraknConfig - Project directory in use: /opt/grakn
2018-08-18 15:15:47,530 [main] INFO  ai.grakn.engine.GraknConfig - Configuration file in use: /opt/grakn/conf/grakn.properties
2018-08-18 15:15:47,852 [main] INFO  ai.grakn.engine.GraknConfig - Project directory in use: /opt/grakn
2018-08-18 15:15:47,852 [main] INFO  ai.grakn.engine.GraknConfig - Configuration file in use: /opt/grakn/conf/grakn.properties
2018-08-18 15:15:57,921 [main] ERROR ai.grakn.engine.bootup.GraknBootup - An error has occurred during boot-up. Please run 'grakn server status' or check the logs located under the 'logs' directory.
ai.grakn.engine.bootup.BootupException: null
        at ai.grakn.engine.bootup.QueueBootup.start(QueueBootup.java:131)
        at ai.grakn.engine.bootup.QueueBootup.startIfNotRunning(QueueBootup.java:68)
        at ai.grakn.engine.bootup.GraknBootup.serverStart(GraknBootup.java:191)
        at ai.grakn.engine.bootup.GraknBootup.server(GraknBootup.java:143)
        at ai.grakn.engine.bootup.GraknBootup.run(GraknBootup.java:130)
        at ai.grakn.engine.bootup.GraknBootup.main(GraknBootup.java:71)


#2

Thank you for reporting the issue. Can you post the terminal output of grakn server start?


#3

O.K. Kindely see below

grakn server start

====================================================================================================
________ _____ _______ __ __ __ __ _______ _______ _____ _______
| __ || _ \ | _ || | / /| \ | | | _ || _ || _ \ | |
| | ||| | | | | | | || | / / | \ | | | | ||| | | || | | | | |
| | ____ | |
| / | |
| || |/ / | | | | | | | | || |
| / | |
___
| ||_ || _ \ | _ || _ \ | _ | | | __ | | | || _ \ | |
| |__| || | \ \ | | | || | \ \ | | \ | | |
| || |
| || | \ \ | |
__
|||| __|| |||| _|__| _| |||||__| _|______|

                                     THE KNOWLEDGE GRAPH

====================================================================================================

Storage is already running
Starting Queue…FAILED!
Unable to start Queue. Process exited with code 1: ''An error has occurred during boot-up. Please run ‘grakn server status’ or check the logs located under the ‘logs’ directory.
null


#4

Sorry that we missed this issue. Are you able to start Grakn in the end?

We have received a report from several people that Grakn won’t work with CentOS 7 as there’s a core dependency which is quite old and therefore won’t work.

I would recommend to use Ubuntu 16.04 if possible.

Best regards,

Ganesh


#5

Hi Ganesh,

Unfortunatly I still failed to start grakn within docker container. The docker image I use is from openjdk:8-jdk, which is based on Debian. I simply googled and know that Ubuntu is based on debian:stretch.

So my question is, does grakn can be run over a Debian docker container?


#6

My docker file FYI.

#####################################################

FROM openjdk:8-jdk

ARG GRAKN_VERSION=1.3.0
ENV GRAKN_HOME=/opt/grakn

RUN mkdir -p $GRAKN_HOME &&
curl -sSL https://github.com/graknlabs/grakn/releases/download/v${GRAKN_VERSION}/grakn-dist-${GRAKN_VERSION}.tar.gz |
tar -C $GRAKN_HOME --strip-components=1 -xzv

ENV PATH=$PATH:$GRAKN_HOME
WORKDIR $GRAKN_HOME

COPY docker-entrypoint /usr/local/bin
RUN chmod a+x /usr/local/bin/docker-entrypoint

EXPOSE 4567

EXPOSE 9160

EXPOSE 48555

ENTRYPOINT [ “docker-entrypoint” ]


#7

We have no official support for it but yes, it should work.

Have you tried this Docker image maintained by one of our community member, by the way: https://github.com/BFergerson/grakn-docker-toolbox


#8

Yes I tried. Actually my dockerfile is exactly the copy from BFergerson. I had tried built a image based on his dockerfile but I still see failure messages as long as I tried to persist the db folder as a volume.


#9

Ok, so the issue is specific to when you’re working with an external / persisted volume, ie., the Docker image works fine when you don’t. Is this correct?


#10

Yes, it is. If you go back to the top of this discussion and follow the 3 steps I described you should be able to reproduce my issue.


#11

@JonyYang got it. Can you run Grakn on a normal machine or VM for now?

We would get around the issue eventually, but at the moment Docker isn’t officially supported.


#12

Thank you @ganesh. I confirm Grakn can run on a normal machine. Looking forward to seeing a better docker solution for latest grakn.


#13

@JonyYang We’ll mark this as a bug on our side. Thank you for reporting it and have fun with Grakn :slight_smile:


#14

Docker originally used [LinuX Containers(LXC), but later switched to runC formerly known as libcontainer , which runs in the same operating system as its host. This allows it to share a lot of the host operating system resources. Also, it uses a layered filesystem AuFS and manages networking.

AuFS is a layered file system, so you can have a read only part and a write part which are merged together. One could have the common parts of the operating system as read only (and shared amongst all of your containers) and then give each container its own mount for writing.

So, let’s say you have a 1 GB container image; if you wanted to use a full VM, you would need to have 1 GB times x number of VMs you want. With Docker and AuFS you can share the bulk of the 1 GB between all the containers and if you have 1000 containers you still might only have a little over 1 GB of space for the containers OS (assuming they are all running the same OS image).

A full virtualized system gets its own set of resources allocated to it, and does minimal sharing. You get more isolation, but it is much heavier (requires more resources). With Docker you get less isolation, but the containers are lightweight (require fewer resources). So you could easily run thousands of containers on a host, and it won’t even blink. Try doing that with Xen, and unless you have a really big host, I don’t think it is possible.

A full virtualized system usually takes minutes to start, whereas Docker/LXC/runC containers take seconds, and often even less than a second.

There are pros and cons for each type of virtualized system. If you want full isolation with guaranteed resources, a full VM is the way to go. If you just want to isolate processes from each other and want to run a ton of them on a reasonably sized host, then Docker/LXC/runC seems to be the way to go.