Convert a serialization config JSON to a OCI runtime configuration.

[s-urbaniak]

As per #82 (comment): Currently, the oci-image-tool supports a simple unpacking mechanism to inspect deflated images and validation.

More concretely the config/image JSON https://github.com/opencontainers/image-spec/blob/master/serialization.md#image-json-description has to be converted to a https://github.com/opencontainers/runtime-spec/blob/master/config.md#container-configuration-file.

[wking]

On Tue, May 24, 2016 at 06:44:43PM -0700, Sergiusz Urbaniak wrote:

More concretely the config/image JSON
https://github.com/opencontainers/image-spec/blob/master/serialization.md#image-json-description
has to be converted to a
https://github.com/opencontainers/runtime-spec/blob/master/config.md#container-configuration-file.

I don't think this is possible. For example, the image-spec schema
currently has ‘ExposedPorts’, but networking is out-of-scope for the
runtime config 1. You'd have to select a particular network manager
(that isn't covered by an OCI spec) and setup hooks to drive that
network manager.

Stepping back, I see no reason for the image-spec repo to care about
what is in the runtime config (see also @stevvooe's “decouple”
wording in 2). I think imaging tools need to know “this is a
directory that I'm (de)serializing”. Then the image spec can add on
distribution-oriented layers like “distribute that directory using
layered tarballs” or “sign name ↔ directory associations”. But baking
knowledge of the filesystem directory (e.g. there's a ‘config.json’
specified by runtime-spec, there may be a ‘rootfs’ directory
referenced from ‘root.path’ in that config.json, …) into the image
spec seems like it's breaking a perfectly good abstraction boundary
for no purpose.

If the “we don't care what's inside the directory” approach from my
last paragraph gains traction, there will still be a need for some
conversion logic for backwards compatibility with existing images.
But I think we want the image spec to get out of that business (and
let the runtimes handle it).

 Subject: Do we have any plan for network and storage specs?
 Date: Fri, 14 Aug 2015 02:36:15 -0700 (PDT)
 Message-Id: <c056925e-6ecf-4457-8c16-7ab34d279604@opencontainers.org>

[philips]

@wking exposed ports is useful metadata from the image. I think it needs to exist to retain useful UX from builder to runner.

I am really really unclear on what you want to see here.

Furthermore, retaining the UX of container runtimes is a desirable part of the spec and explicitly called out in the README and image format spec project proposal:

docker run example.com/org/app:v1.0.0
rkt run example.com/org/app,version=v1.0.0

[wking]

On Wed, May 25, 2016 at 08:22:16AM -0700, Brandon Philips wrote:

@wking exposed ports is useful metadata from the image. I think it
needs to exist to retain useful UX from builder to runner.

Then I think you want to file a PR against runtime-spec (or a
networking OCI spec?) to get it supported. I don't think image-spec
is a good place to specify container networking.

I am really really unclear on what you want to see here.

As I understand it, #82 handles unpacking a local directory from layer
tarballs. I want to see image-spec drop
application/vnd.oci.image.serialization.config.v1+json, and have
runtime-spec claim a MIME type for config.json. Then
application/vnd.oci.image.manifest.v1+json is just a list of layers
(and optional annotations). Then runtime-spec can wander around
however it likes and image-spec would still be able to distribute the
new configs without changes. And folks could add alternative runtime
configs, and image-spec could distribute them without changes. And I
could package up a non-container package (e.g. a Gentoo stage3) and
use image-spec tooling to distribute that too (although I obviously
wouldn't want to point runC at the config.json-less unpacked
directory).

Teaching image-spec about what's inside the distributed directory
pokes holes in the runtime's abstraction, and defining a local config
that has to be translated to a runtime-spec config pokes big holes in
the abstraction. Sometimes that sort of abstraction-breaking is
useful, but in this case I don't see an upside, and I see flexibility
and maintenance downsides.

[philips]

@wking the runtime can IGNORE this information and it can be used by a higher level system. I will file an issue about how to convert it but I don't think dropping it makes sense.

[philips]

@wking I disagree with your suggestion of the split. The image format needs a way of expressing runtime defaults that aren't host specific which the runtime spec has not tackled and made difficult by recombining everything into a single config.json.

[wking]

On Wed, May 25, 2016 at 09:11:49AM -0700, Brandon Philips wrote:

the runtime can IGNORE this information and it can be used by a
higher level system.

And I'm fine with that, although I think we want to be careful about
how external tools like network managers are included [1]. I just
don't think the image-spec should be in the business of orchestrating
this. You can have:

• Orchestration layer
  • Call image-spec tooling to fetch a generic directory by name
  • Call network-setup tooling to convert a config file's ExposedPorts
    entries to libnetwork hooks (or some such).
  • Call ocitools to translate fromContainer entries in the config.
  • … Call other tools to localize the config …
  • Call runtime-spec tooling (e.g. runC) to launch the adapted config.

I think that such a narrowly-scoped image-spec would be easier to
maintain and use than a broadly-scoped image-spec like:

• Orchestration layer
  • image-spec
    • Call image-spec tooling to fetch a generic directory by name
    • Call network-setup tooling to convert a config file's
      ExposedPorts entries to libnetwork hooks (or some such).
    • Call ocitools to translate fromContainer entries in the config.
    • … Call other tools to localize the config …
  • Call runtime-spec tooling (e.g. runC) to launch the adapted config.

With the narrowly-scoped image-spec, you can add as many localization
steps to your orchestration layer as you like, and don't have to file
image-spec PRs for “please have ocitools translate fromContainer”,
etc.

On Wed, May 25, 2016 at 09:13:45AM -0700, Brandon Philips wrote:

The image format needs a way of expressing runtime defaults that
aren't host specific which the runtime spec has not tackled and made
difficult by recombining everything into a single config.json.

The orchestration layer needs to do that sort of thing, but I don't
see how it benefits from being tied into image-spec (which is about
efficiently and securely shipping directories between hosts).