Friday, December 22, 2017

Ubuntu apt not able to fetch updates or install packages from behind a proxy


Apt can take the system environment values of the http_proxy as well as https_proxy defined.


So to get over this set the proxies as the user environment variables or  system environment variables, which will be automatically used by apt actions

So set env variables related to the http_proxy or https_proxy these in either of the following as 

~/.profile
~/.bash_profile
~/.bashrc

or 

If this has to be set as the system Env variables, these can be set in the file as 

/etc/profile

or 

/etc/environment


or in a file in the 

/etc/profile.d 

like /etc/profile.d/more_env_variables.sh


The entries for the proxy are as 


export http_proxy=http://<Proxy Server FQDN or IP:<Port>

export https_proxy=http://<Proxy Server FQDN or IP>:<Port>

export no_proxy=<IP1|FQDN1| ...>


To be noted that as these files are executed at the time of login the user may have to logoff and logback in or can execute the Profile Variable files as 

. /bash_profile 

please note the [SPACE] after the dot '.' .

Alternative way is that if there is a particular proxy set for APT or  you just want the proxy server to be reached by APT then you can specify the proxy server in /etc/apt/apt.conf 

Or depending on the Ubuntu distribution version you can create a file under /etc/apt/apt.conf.d

to have the contents like this 

Acquire::http::proxy "http://<Proxy_server_FQDN_or_IP>:<PORT>
Acquire::https::proxy "http://<Proxy_server_FQDN_or_IP<:<PORT>

Replace the values above with the Proxy server IP and Port being used at your network.

An Alternative way if you do not want the proxy settings to be saved persistently just want to be available for the current session then, simply export the env variables as below


export http_proxy=http://<Proxy_server_FQDN_or_IP>:<PORT>
export https_proxy=http://<Proxy_server_FQDN_or_IP>:<PORT>





Docker daemon not able to download images from behind a proxy. Enable docker deamon to download images from Docker Hub via a proxy server


This happens as the docker daemon tries to reach direct to the Docker Hub on the internet but the security requires that all traffic to the internet will be allowed only via PROXY server 



If not sure what is the configuration file for the docker service in terms of systemd, you can get the systemd docker configuration file using 'systemctl status docker' or 'systemctl status docker.service'.

The configuration file is highlighted as below

[root@rally docker.service.d]# systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2017-12-23 04:55:19 EST; 13min ago
     Docs: https://docs.docker.com
 Main PID: 22381 (dockerd)
   CGroup: /system.slice/docker.service
           ├─22381 /usr/bin/dockerd
           └─22387 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=...



Stop the docker service

systemctl stop docker



Edit the file  /usr/lib/systemd/system/docker.service



Make entries in the [service] section and put the 'Environment' variables for HTTP_PROXY and HTTPS_PROXY as per the proxy server type you have through which the docker daemon will try to reach out the Docker Hub to get the docker images.

If there are certain IPs that is needed that if ever docker daemon wants to reach to them not going through the proxy, put the entries of such IPs and FQDN in the 'Environment' variable as 'NO_PROXY'

For more on the related syntax please see the excerpt from the file /usr/lib/systemd/system/docker.service


[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID

Environment="HTTP_PROXY=http://<PROXY_Server_NAME_OR_IP>:<PROXY_PORT>"
Environment="HTTP_PROXY=http://<PROXY_Server_NAME_OR_IP>:<PROXY_PORT>"
Environment="NO_PROXY=<FQDN1|IPAddr1|IPAddr2|FQDN2| ...>"

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s

[Install]
WantedBy=multi-user.target



Save and exit the file 

Issue a systemctl daemon-reload so as to acknowledge that the docker systemd file has changed on the disk

Restart/start  the docker service

systemctl daemon-reload
systemctl restart docker