Rosetta emulation changes shell behaviour

[dometto]

Description

Enabling Rosetta emulation causes standard ways of obtaining a login shell (as opposed to a non-login shell) on Linux containers to fail.

su - (as opposed to su) should normally yield a login shell, i.e. a shell that executes the default profile scripts for the logged-in user (for bash, this is /etc/profile).

However, with Rosetta enabled, running su - (or even /usr/bin/login) on Docker containers yields a non-login shell. This means /etc/profile does not get executed for the logged-in user.

Note that this only happens with an amd64 container image using Rosetta emulation. When disabling Rosetta or using an arm64 image (on Apple Silicon), everything works as expected.

The issue seems to be caused by the way in which the /rosetta/rosetta command is prepended to processes (also see #6794). A shell is a login shell if $0 starts with a '-'. Locally, this looks as follows:

$ echo $0
-bash

It seems like with Rosetta emulation, the -bash argument is getting swallowed, while this doesn't happen with qemu emulation.

With Rosetta:

% docker run --rm -it --platform linux/amd64 ubuntu:jammy /usr/bin/su - root            
root@83d14c6c301b:~# echo $0
/bin/bash
root@83d14c6c301b:~# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 06:18 pts/0    00:00:00 /rosetta/rosetta /usr/bin/su - root
root         7     1  0 06:18 pts/0    00:00:00 /rosetta/rosetta /bin/bash
root        11     7  0 06:18 pts/0    00:00:00 /usr/bin/ps -ef

Without Rosetta, the needed -bash argument is present:

root@3aa7b6bee2d6:~# echo $0
-bash
root@3aa7b6bee2d6:~# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 06:19 pts/0    00:00:00 /usr/bin/qemu-x86_64 /usr/bin/su /usr/bin/su - root
root         8     1  1 06:19 pts/0    00:00:00 /usr/bin/qemu-x86_64 /bin/bash -bash
root        23     8  0 06:19 ?        00:00:00 ps -ef

Reproduce

In bash, we can use shopt to determine whether we have a login shell.

Using an amd image with Rosetta enabled:

% docker run --rm -it --platform linux/amd64 ubuntu:jammy /usr/bin/su - root -c shopt | grep login_shell
login_shell    	off

Expected behavior

Using an amd image with Rosetta disabled, everything works fine:

% docker run --rm -it --platform linux/amd64 ubuntu:jammy /usr/bin/su - root -c shopt | grep login_shell
login_shell    	on

Or with an arm image:

% docker run --rm -it --platform linux/arm64 ubuntu:jammy /usr/bin/su - root -c shopt | grep login_shell
login_shell    	on

docker version

Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.21.0
 Git commit:        ed223bc820
 Built:             Thu Aug 31 17:24:32 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.23.0 (120376)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:36 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    24.0.6
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.4
    Path:     /Users/3060845/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0-desktop.1
    Path:     /Users/3060845/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/3060845/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/3060845/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.7
    Path:     /Users/3060845/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/3060845/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/3060845/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.24.1
    Path:     /Users/3060845/.docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.3.13-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 5
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: 2f6c2701-c3a5-4afe-9872-4ecea81f94f8
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

531FFE47-461D-4B23-A111-41FFEF3628ED/20231003062409

Additional Info

No response

[JasringStw]

I see the exact same behaviour, I have an image I run on my Intel and Apple Silicon Mac but the emulated version I always have to set my variables myself because the profile is not loaded.

[JasringStw]

Seems to be solved in the latest update of Docker Desktop (4.25). for me, it's working fine at least, on M1 and MacOS Sonoma 14.1

[dgageot]

This is fixed in 4.25