This took me some time. The problem was occurring only on https, not on http. So ruby code, gems, versions were fine. Turns out I needed this line in my apache config:

RequestHeader set X_FORWARDED_PROTO 'https' env=HTTPS

Actually, this worked better:

RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}

For nginx it is something similar:

proxy_set_header X-Forwarded-Proto https;