Can't build from source nor run from DockerHub

[marcellodesales]

I'm trying to run from DockerHub and I cannot generate anything... I have Docker 1.2 installed on my server...

# root at pppdc9prd5pq in /tmp/docker-image-graph on git:master o [22:38:06]
$  docker info
Containers: 61
Images: 638
Storage Driver: devicemapper
 Pool Name: docker-0:19-142439779-pool
 Pool Blocksize: 64 Kb
 Data file: /docker/lib/devicemapper/devicemapper/data
 Metadata file: /docker/lib/devicemapper/devicemapper/metadata
 Data Space Used: 10936.1 Mb
 Data Space Total: 102400.0 Mb
 Metadata Space Used: 30.3 Mb
 Metadata Space Total: 2048.0 Mb
Execution Driver: native-0.2
Kernel Version: 2.6.32-431.1.2.el6.x86_64
Operating System: <unknown>

# root at pppdc9prd5pq in /tmp/docker-image-graph on git:master o [22:41:43]
$ docker --version
Docker version 1.2.0, build fa7b24f/1.2.0

Generate the local image

What version is this talking about?

 root at pppdc9prd5pq in /export/home/mdesales [22:29:43]
$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
>   centurylink/image-graph > docker_images.png
/usr/lib/ruby/gems/2.1.0/gems/docker-api-1.21.2/lib/docker/connection.rb:46:in `rescue in request': client and server don't have same version (client : 1.16, server: 1.14) (Docker::Error::NotFoundError)
    from /usr/lib/ruby/gems/2.1.0/gems/docker-api-1.21.2/lib/docker/connection.rb:38:in `request'
" ============================================================================
    from /usr/lib/ruby/gems/2.1.0/gems/docker-api-1.21.2/lib/docker/connection.rb:65:in `block (2 levels) in <class:Connection>'
    from /usr/lib/ruby/gems/2.1.0/gems/docker-api-1.21.2/lib/docker/image.rb:168:in `all'
    from ./image-graph-cmd.rb:8:in `<main>'

Generating through the server...

Running is not working... Can you guys be more clear about the requirements? Is that the latest Docker?

Trying to build from github

# root at pppdc9prd5pq in /tmp/docker-image-graph on git:master o [22:36:33]
$ docker build -t graph .
Sending build context to Docker daemon 1.103 MB
Sending build context to Docker daemon
Step 0 : FROM alpine:3.1
Pulling repository alpine
878b6301beda: Download complete
 ---> 878b6301beda
Step 1 : MAINTAINER CenturyLink Labs <clt-labs-futuretech@centurylink.com>
 ---> Running in ec9105e61f64
 ---> b03461bc08d1
Removing intermediate container ec9105e61f64
Step 2 : ENTRYPOINT ["/usr/src/app/image-graph.sh"]
 ---> Running in 1d181865714d
 ---> 32f3c2febd53
Removing intermediate container 1d181865714d
Step 3 : CMD [""]
 ---> Running in 6e6985b8baf8
 ---> 4d7f9a0cd358
Removing intermediate container 6e6985b8baf8
Step 4 : RUN apk update && apk add ruby-dev graphviz ttf-ubuntu-font-family ca-certificates
 ---> Running in 392e6727ad2e
fetch http://dl-4.alpinelinux.org/alpine/v3.1/main/x86_64/APKINDEX.tar.gz
ERROR: http://dl-4.alpinelinux.org/alpine/v3.1/main: IO ERROR
WARNING: Ignoring APKINDEX.689bb31a.tar.gz: No such file or directory
1 errors; 15 distinct packages available
2015/06/26 22:37:10 The command [/bin/sh -c apk update && apk add ruby-dev graphviz ttf-ubuntu-font-family ca-certificates] returned a non-zero code: 1

thanks for any help!
Marcello

[bdehamer]

@marcellodesales Thanks for your question. My only thought is that this is an issue with the version of Docker that you are using. I'm using 1.5.0 here:

$ docker --version
Docker version 1.5.0, build a8a31ef-dirty

Under 1.5, I was able to run the image without problem. Didn't receive any issues when building either:

 $ docker build -t graph .
Sending build context to Docker daemon 1.186 MB
Sending build context to Docker daemon 
Step 0 : FROM alpine:3.1
Pulling repository alpine
878b6301beda: Download complete 
Status: Downloaded newer image for alpine:3.1
 ---> 878b6301beda
Step 1 : MAINTAINER CenturyLink Labs <clt-labs-futuretech@centurylink.com>
 ---> Running in 4403266bd24b
 ---> f61733501978
Removing intermediate container 4403266bd24b
Step 2 : ENTRYPOINT /usr/src/app/image-graph.sh
 ---> Running in 12199f44770c
 ---> a716b874fd05
Removing intermediate container 12199f44770c
Step 3 : CMD 
 ---> Running in d809fc7167ae
 ---> 2911205360ee
Removing intermediate container d809fc7167ae
Step 4 : RUN apk update && apk add ruby-dev graphviz ttf-ubuntu-font-family ca-certificates
 ---> Running in ee7707326432
fetch http://dl-4.alpinelinux.org/alpine/v3.1/main/x86_64/APKINDEX.tar.gz
v3.1.4-61-g56eabef [http://dl-4.alpinelinux.org/alpine/v3.1/main]
OK: 4864 distinct packages available
(1/47) Installing libffi (3.0.13-r0)
(2/47) Installing gdbm (1.11-r0)
(3/47) Installing ncurses-terminfo-base (5.9-r3)
(4/47) Installing ncurses-libs (5.9-r3)
(5/47) Installing readline (6.3-r3)
(6/47) Installing yaml (0.1.6-r1)
(7/47) Installing ruby-libs (2.1.5-r1)
(8/47) Installing ruby (2.1.5-r1)
(9/47) Installing ruby-doc (2.1.5-r1)
(10/47) Installing ruby-irb (2.1.5-r1)
(11/47) Installing ruby-json (2.1.5-r1)
(12/47) Installing ruby-rdoc (2.1.5-r1)
(13/47) Installing ruby-rake (2.1.5-r1)
(14/47) Installing ruby-bigdecimal (2.1.5-r1)
(15/47) Installing ruby-io-console (2.1.5-r1)
(16/47) Installing ruby-minitest (2.1.5-r1)
(17/47) Installing pkgconf (0.9.7-r0)
(18/47) Installing pkgconfig (0.25-r1)
(19/47) Installing ruby-dev (2.1.5-r1)
(20/47) Installing libxau (1.0.8-r0)
(21/47) Installing libxdmcp (1.1.1-r0)
(22/47) Installing libxcb (1.11-r0)
(23/47) Installing libx11 (1.6.2-r1)
(24/47) Installing libxrender (0.9.8-r0)
(25/47) Installing libxext (1.3.3-r0)
(26/47) Installing expat (2.1.0-r1)
(27/47) Installing libpng (1.6.16-r0)
(28/47) Installing freetype (2.5.4-r0)
(29/47) Installing fontconfig (2.11.1-r1)
(30/47) Installing pixman (0.32.6-r0)
(31/47) Installing cairo (1.14.0-r0)
(32/47) Installing libgcc (4.8.3-r0)
(33/47) Installing libintl (0.19.3-r0)
(34/47) Installing glib (2.42.0-r0)
(35/47) Installing libltdl (2.4.3-r0)
(36/47) Installing libxft (2.3.2-r0)
(37/47) Installing harfbuzz (0.9.36-r0)
(38/47) Installing pango (1.36.8-r0)
Executing pango-1.36.8-r0.post-install
(39/47) Installing libstdc++ (4.8.3-r0)
(40/47) Installing graphviz (2.38.0-r2)
(41/47) Installing ttf-ubuntu-font-family (0.80-r0)
(42/47) Installing run-parts (4.4-r0)
(43/47) Installing openssl (1.0.1o-r0)
(44/47) Installing lua5.2-libs (5.2.3-r0)
(45/47) Installing lua5.2 (5.2.3-r0)
(46/47) Installing lua5.2-posix (32-r1)
(47/47) Installing ca-certificates (20141019-r0)
Executing busybox-1.22.1-r15.trigger
Executing fontconfig-2.11.1-r1.trigger
Executing glib-2.42.0-r0.trigger
Executing ca-certificates-20141019-r0.trigger
OK: 177 MiB in 62 packages
 ---> e71a48c73f74
Removing intermediate container ee7707326432
Step 5 : RUN gem install --no-rdoc --no-ri docker-api sinatra
 ---> Running in 149a06dbeab9
Successfully installed excon-0.45.3
Successfully installed docker-api-1.21.4
Successfully installed rack-1.6.4
Successfully installed rack-protection-1.5.3
Successfully installed tilt-2.0.1
Successfully installed sinatra-1.4.6
6 gems installed
 ---> 96c734f9db59
Removing intermediate container 149a06dbeab9
Step 6 : RUN dot -c
 ---> Running in adcc0453c075
 ---> 5e37fbe161e4
Removing intermediate container adcc0453c075
Step 7 : ADD . /usr/src/app/
 ---> a11bd52fa799
Removing intermediate container a2f7b8711983
Step 8 : WORKDIR /usr/src/app/
 ---> Running in f987bd1cc44d
 ---> a00551ec033d
Removing intermediate container f987bd1cc44d
Successfully built a00551ec033d

I'll try and spin-up a Docker 1.2 environment and see if I can re-pro your issue.

[marcellodesales]

Hi @bdehamer

I upgraded my server to 1.6.2 and it's working...

# root at pppdc9prd5pq in /export/home/mdesales [15:41:23]
$ docker logs -f 26134901465c62559edfc345bc3cf92bd0b8ff313d8ebd746856ddcc3a0cd58b
[2015-06-29 22:41:24] INFO  WEBrick 1.3.1
[2015-06-29 22:41:24] INFO  ruby 2.1.5 (2014-11-13) [x86_64-linux-musl]
== Sinatra (v1.4.6) has taken the stage on 3000 for development with backup from WEBrick
[2015-06-29 22:41:24] INFO  WEBrick::HTTPServer#start: pid=9 port=3000
172.29.33.12 - - [29/Jun/2015:22:41:47 +0000] "GET / HTTP/1.1" 200 3703 0.0696
172.29.33.12 - - [29/Jun/2015:22:41:47 UTC] "GET / HTTP/1.1" 200 3703
- -> /
172.29.33.12 - - [29/Jun/2015:22:41:49 +0000] "GET /texture-noise.png HTTP/1.1" 200 25768 0.0006
172.29.33.12 - - [29/Jun/2015:22:41:49 UTC] "GET /texture-noise.png HTTP/1.1" 200 25768
http://pppdc9prd5pq:3000/ -> /texture-noise.png

However, there is no image being served... The screen still shows the empty information... I have confirmed that I have docker running...

# root at pppdc9prd5pq in /export/home/mdesales [15:41:02]
$ ls -la /var/run/docker.sock
srw-rw---- 1 root docker 0 Jun 27 09:22 /var/run/docker.sock

# root at pppdc9prd5pq in /export/home/mdesales [15:40:45]
$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock \
  -e PORT=3000 -p 3000:3000 centurylink/image-graph

Can you let me know if that's properly set?

[bdehamer]

Hmm. Now I'm wondering if there isn't a browser problem. My log output here looks like this:

[2015-06-29 23:28:42] INFO  WEBrick 1.3.1
[2015-06-29 23:28:42] INFO  ruby 2.1.5 (2014-11-13) [x86_64-linux-musl]
== Sinatra (v1.4.6) has taken the stage on 3000 for development with backup from WEBrick
[2015-06-29 23:28:42] INFO  WEBrick::HTTPServer#start: pid=9 port=3000
10.0.2.2 - - [29/Jun/2015:23:28:52 +0000] "GET / HTTP/1.1" 200 3703 0.0042
10.0.2.2 - - [29/Jun/2015:23:28:52 UTC] "GET / HTTP/1.1" 200 3703
- -> /
10.0.2.2 - - [29/Jun/2015:23:28:53 +0000] "GET /texture-noise.png HTTP/1.1" 200 25768 0.0007
10.0.2.2 - - [29/Jun/2015:23:28:53 UTC] "GET /texture-noise.png HTTP/1.1" 200 25768
http://localhost:8080/ -> /texture-noise.png
10.0.2.2 - - [29/Jun/2015:23:28:53 +0000] "GET /images.json HTTP/1.1" 200 2265 0.0319
10.0.2.2 - - [29/Jun/2015:23:28:53 UTC] "GET /images.json HTTP/1.1" 200 2265
http://localhost:8080/ -> /images.json

Note the call to /images.json. If things are working properly, there should actually be a lot of calls to /images.jsonas the client-side javascript polls the back-end looking for changes.

Are you seeing any errors in the browser? Maybe in the developer tools console?

[bdehamer]

I should probably point-out that we have a hosted version of this same tool now that is much easier to use:

https://imagelayers.io/

The primary difference is that the imagelayers.io tool allows you to inspect images on the Docker Hub instead of images in your local image cache.

[marcellodesales]

I'm aware of imagelayers.io, and I forked in the hopes to generate a hosted version of it... Couldn't get Ruby compiled with Node.js in Alpine yet... So, now I just want a quick way to visualize the images in a given host as we have an Internal registry...

The current problem is on the server I guess :(

[bdehamer]

@marcellodesales it does look like it might be something on the server side. Can you paste the output of running docker images on your system. There may be one image in particular that is tripping it up and I'd love to be able to replicate the issue on my system. Thanks.

[marcellodesales]

@bdehamer here it is!

# root at pppdc9prd5pq in /export/home/mdesales [8:17:01]
$ docker ps
CONTAINER ID        IMAGE                                 COMMAND                CREATED             STATUS              PORTS                                                NAMES
26134901465c        centurylink/image-graph:latest        "/usr/src/app/image-   5 days ago          Up 5 days           0.0.0.0:3000->3000/tcp                               mad_tesla
00fd1ef818cc        quay.io/coreos/registry:latest        "/sbin/my_init"        7 days ago          Up 7 days           0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 8443/tcp   coreos-registry-server
a9e319b2c50f        marvambass/versatile-postfix:latest   "/opt/startup.sh doc   4 months ago        Up 7 days           0.0.0.0:537->25/tcp                                  coreos-registry-email-server
a543888b4c84        mysql:latest                          "/entrypoint.sh mysq   8 months ago        Up 7 days           0.0.0.0:3306->3306/tcp                               coreos-registry-mysql-server
a7d47c9d9ba7        quay.io/quay/redis:latest             "/usr/bin/redis-serv   8 months ago        Up 7 days           0.0.0.0:6379->6379/tcp                               coreos-registry-redis-server
ff341e697a80        busybox:latest                        "/bin/sh"              8 months ago        Up 7 days                                                                coreos-registry-mysql-data

[bdehamer]

@marcellodesales can I see your docker images output instead of docker ps. The image-graph tool doesn't really care what's running, it only looks at the images in your local Docker image cache. Thanks.

[marcellodesales]

@bdehamer I'm really sorry :) I might have been sleepy :P Here they are...

[mdesales@pppdc9prd5pq ~]$ sudo docker images
Enter your Intuit  password:
REPOSITORY                                                       TAG                   IMAGE ID            CREATED             VIRTUAL SIZE
<none>                                                           <none>                4d7f9a0cd358        9 days ago          5.041 MB
quay.io/coreos/registry                                          keystone-test         6b463c0bd34d        10 days ago         827.3 MB
quay.io/coreos/registry                                          latest                82a0e77e6523        12 days ago         785.4 MB
<none>                                                           <none>                2505ffcd659c        2 weeks ago         785.6 MB
alpine                                                           3.1                   878b6301beda        3 weeks ago         5.041 MB
dckr/docker-backup                                               latest                f0abd411855c        8 weeks ago         13.15 MB
<none>                                                           <none>                740f4e47c3da        8 weeks ago         762 MB
centurylink/image-graph                                          latest                79c89340e7f2        10 weeks ago        132.8 MB
quay.io/coreos/registry                                          previous              9ef3facd64da        3 months ago        775.8 MB
centurylink/openssl                                              latest                0825ae8b37af        4 months ago        5.589 MB
marvambass/versatile-postfix                                     latest                97dbbf785c58        5 months ago        280.8 MB
<none>                                                           <none>                9b94836a92fd        5 months ago        766 MB
<none>                                                           <none>                90454ceed462        6 months ago        770.6 MB
quay.io/coreos/registry                                          githubfix             715582859e60        6 months ago        770.7 MB
<none>                                                           <none>                ca51b144393f        6 months ago        770.6 MB
<none>                                                           <none>                b8871899eea1        7 months ago        755.4 MB
fabiana/app                                                      with-version          dfad8f4f59af        7 months ago        2.434 MB
<none>                                                           <none>                4ab24fd0fc51        7 months ago        2.434 MB
<none>                                                           <none>                f7e8fa73f244        7 months ago        2.434 MB
marcello/app                                                     latest                a7ecb3d979f1        7 months ago        2.433 MB
fabiana/app                                                      latest                f2c040e711b8        7 months ago        2.433 MB
<none>                                                           <none>                1ae9e4a28188        8 months ago        750.9 MB
<none>                                                           <none>                084ace1551c6        8 months ago        750.9 MB
coreos/mysqldata                                                 latest                93e3036d9b24        8 months ago        2.433 MB
dockerhub.corp.intuit.net/marcellodesales/mysql-data-container   latest                93e3036d9b24        8 months ago        2.433 MB
<none>                                                           <none>                5f2ed4144547        8 months ago        750.8 MB
mysql                                                            latest                3c6d7e5c8c1b        8 months ago        235.5 MB
quay.io/coreos/registry                                          ea96dbb               e13b05e00e62        8 months ago        745 MB
platform.registry.docker.corp.intuit.net/editor/cloud9           nodejs                47875688379c        8 months ago        817.1 MB
busybox                                                          ubuntu-14.04          d940f6fef591        9 months ago        5.609 MB
busybox                                                          ubuntu-12.04          d415c60e5ea3        9 months ago        5.455 MB
busybox                                                          buildroot-2014.02     e72ac664f4f0        9 months ago        2.433 MB
busybox                                                          latest                e72ac664f4f0        9 months ago        2.433 MB
dockerhub.corp.intuit.net/marcellodesales/busybox                latest                e72ac664f4f0        9 months ago        2.433 MB
busybox                                                          buildroot-2013.08.1   4b2909447dbe        9 months ago        2.489 MB
catatnight/postfix                                               latest                deb60ba9b476        9 months ago        256.6 MB
quintenk/tomcat                                                  7                     5b483d8d82d6        13 months ago       917 MB
quay.io/quay/redis                                               latest                c89063721e41        13 months ago       423 MB

[bdehamer]

@marcellodesales I'm still in the process of troubleshooting this (thanks for your patience). If possible, could you paste me the result of hitting the /images.json endpoint in your browser (once you've started the image-graph container). This is the raw JSON output that is consumed by the client-side JavaScript which is responsible for rendering the image graph. I suspect that there is some unexpected value in this output that is causing the problem.

It should look something like this:

[  
   [  
      {  
         "v":"31f630c65071968699d327be41add2e301d06568a4914e1aa67c98e1db34a9d8",
         "f":"31f630c65071 — 5 MBalpine:latest"
      },
      "",
      "ADD file:98d5decf83ee59e1dd63cd038be33a43e840fa4bd2aca68cb90f20ea06527c40 in /"
   ],
   [  
      {  
         "v":"742f35925441019dbe83762c071017eec8148e756161a33aa2030cc7743c42e5",
         "f":"742f35925441 — 263 MBmarvambass/versatile-postfix:latest"
      },
      "3f2885e09ccd7209724ea189a100af2b849001b3d2f2552060d28ed4098e1b7c",
      "CMD [\"-h\"]"
   ],
. . .

Thanks.

[marcellodesales]

@bdehamer So, the output is extremely long... It also took A LONG TIME to return... Here's the output of the second call...

Marcello-New2015:~ mdesales$ time curl http://pppdc9prd5pq:3000/images.json
real    1m46.749s
user    0m0.006s
sys 0m0.012s

I hope you can find clues from the output below... Or maybe because it is too long?

JSON

<elided>

[bdehamer]

@marcellodesales The JSON you sent was super helpful. There are definitely a few entries in there that are causing the rendering problem:

  [
      {
         "v":"dfad8f4f59af080df6173f7c8cd246c8c8191a3bf060e32acd4aca2f425ade1b",
         "f":"dfad8f4f59af - 2 MB<span class=\"tags\">fabiana/app:with-version</span>"
      },
      "73b8baa1132dfe3cf2f059a708d0dfebb4347d610407ee3fb9496686958e5df2",
      [
         "sh"
      ]
   ],
   [
      {
         "v":"4ab24fd0fc5123209e94b966e9420ac206a0edb909d6b016fbbee869e043f13c",
         "f":"4ab24fd0fc51 - 2 MB<span class=\"tags\"></span>"
      },
      "73b8baa1132dfe3cf2f059a708d0dfebb4347d610407ee3fb9496686958e5df2",
      [
         "sh"
      ]
   ],
   [
      {
         "v":"73b8baa1132dfe3cf2f059a708d0dfebb4347d610407ee3fb9496686958e5df2",
         "f":"73b8baa1132d - 2 MB<span class=\"tags\"></span>"
      },
      "f2c040e711b839644e0ee0252ec523d0dd9a850d51d7532922cefcebf78615e4",
      [
         "sh"
      ]
   ],
   [
      {
         "v":"f7e8fa73f244a624a6777b8071d25aca05cfae84f0b898b0422f296be1dbb561",
         "f":"f7e8fa73f244 - 2 MB<span class=\"tags\"></span>"
      },
      "f2c040e711b839644e0ee0252ec523d0dd9a850d51d7532922cefcebf78615e4",
      [
         "sh"
      ]
   ],
   [
      {
         "v":"a7ecb3d979f1ef89a7ee128f3e3ba5a76c3672899fa61f8f3409fd2bfddcfd7f",
         "f":"a7ecb3d979f1 - 2 MB<span class=\"tags\">marcello/app:latest</span>"
      },
      "e72ac664f4f0c6a061ac4ef332557a70d69b0c624b6add35f1c181ff7fff2287",
      [
         "sh"
      ]
   ],
   [
      {
         "v":"f2c040e711b839644e0ee0252ec523d0dd9a850d51d7532922cefcebf78615e4",
         "f":"f2c040e711b8 - 2 MB<span class=\"tags\">fabiana/app:latest</span>"
      },
      "e72ac664f4f0c6a061ac4ef332557a70d69b0c624b6add35f1c181ff7fff2287",
      [
         "sh"
      ]
   ],

Something about the data in the fabiana/app and marcello/app images is resulting in some odd data in the JSON output.

After removing those images, I was able to get the rest of the data to render without problem:

I'll see if I can't figure out what it is about those images that is causing a problem and try to make a fix.

[bdehamer]

@marcellodesales

I just pushed a new centurylink/image-graph image with a small fix to the server-side code that I think may address the issue you are seeing. Please pull it down and let me know if it resolves your issue.

[marcellodesales]

@bdehamer Perfect! I will double-check that... Ahh, those images were created for a tech session I gave last year... super old :) Not sure what's going on there, but I will remove them... I will check the fix first, and then remove the image and report back to you!

thanks a lot for the help!

[bdehamer]

It looks like those images were maybe built without using a Dockerfile (i.e. doing a docker run and then a docker commit). When you build images in that way the metadata for the layer is stored a little differently and that is what was causing problems for my script.