Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Imagetools multiple repositories #1137
Conversation
|
Could we add some progress logs so the user can see what is going on when layer transfers happen? Ideally, we could just reuse the build progressbar and show that transfers the same way we see on image pulls during build. For now, just printing logs like "copying blob sha256: from .. to .. & done` would do as well. I don't this currently does cross-repo mounting but it should be quite easy to add support for it. Supposedly the label for the origin of the blob gets missing somewhere. https://github.com/containerd/containerd/blob/main/remotes/handlers.go#L330 |
jedevc
force-pushed
the
imagetools-multiple-repositories
branch
from
7e004d2 to
07b2bec
Compare
|
Have got the cross-repo mounting working now in a Let me know, and I can actually open a PR on buildkit, instead of hacking around with the |
|
Yes, go ahead with the PR to buildkit. |
jedevc
force-pushed
the
imagetools-multiple-repositories
branch
3 times, most recently
from
4401fa4 to
82ef8ab
Compare
jedevc
force-pushed
the
imagetools-multiple-repositories
branch
from
82ef8ab to
26d9f7a
Compare
|
We should review this example in build-push-action repo when 0.9 is out with this PR: https://github.com/docker/build-push-action/blob/master/docs/advanced/copy-between-registries.md |
jedevc
force-pushed
the
imagetools-multiple-repositories
branch
from
26d9f7a to
0602be0
Compare
|
Have rebased onto master, this should be ready to review/merge. |
| var repo string | ||
| for r := range repos { | ||
| repo = r | ||
| var defaultRepo *string |
There was a problem hiding this comment.
The repos map looks unused now. You can just use the earlier range tags/srcs to determine the defaultRepo
There was a problem hiding this comment.
repos is only used to determine the defaultRepo. I think some sort of collection of all the repo names is required, since we only derive a default if all the other repo names present are the same.
We could potentially have a bool to track if we found a different repo that doesn't match the previously selected default during iteration, but I think that's potentially more complex than just tracking all the found repo names and then having a len().
| eg.Go(func() error { | ||
| return progress.Wrap(fmt.Sprintf("pushing %s", t.String()), pw.Write, func(sub progress.SubLogger) error { | ||
| eg2, _ := errgroup.WithContext(ctx) | ||
| for _, s := range srcs { |
There was a problem hiding this comment.
Shouldn't this only happen when src and destination repo are actually different?
This patch modifies the existing combining code in imagetools create to provide better support for multiple repositories down the road. Specifically, the code should no longer rely on a single repository being used for all sources and tags, and should resolve descriptors in their relevant repositories. Signed-off-by: Justin Chadwell <me@jedevc.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
jedevc
force-pushed
the
imagetools-multiple-repositories
branch
from
0602be0 to
f1a9f91
Compare
3 participants
Fixes #486
This PR introduces support for multiple repositories: this allows creating manifest lists with manifests pulled from multiple other locations.
This is split into two commits:
srcstruct public to allow keeping the oci descriptor next to where it can be fetched from.