Proxy servers act as an intermediary for requests from clients seeking resources from other servers. A client connects to the proxy server, requesting some service or available resource from a different server, and the proxy server evaluates the request as a way to simplify and control its complexity.
Note:
Using SSL offloading or using a proxy that handles SSL for Tower is supported. The proxy/load balancer needs to be configured to pass the remote host information.
When offloading SSL to the load balancer or proxy, set nginx_disable_https=true as an extra variable passed to the setup playbook. Refer to um_setup_playbook for information on applying extra variables to the setup playbook.
Sessions in Tower associate an IP address upon creation. Tower policy requires that any use of the session match the original associated IP address.
To provide proxy server support, Tower handles proxied requests (such as ELB in front of Tower, HAProxy, Squid, and tinyproxy) via the REMOTE_HOST_HEADERS
list variable in Tower settings (/etc/tower/conf.d/remote_host_headers.py
). By default REMOTE_HOST_HEADERS
is set to ['REMOTE_ADDR', 'REMOTE_HOST']
.
To enable proxy server support, setup REMOTE_HOST_HEADERS
like the following: REMOTE_HOST_HEADERS = ['HTTP_X_FORWARDED_FOR', 'REMOTE_ADDR', 'REMOTE_HOST']
Note
A new installation of Ansible Tower will not contain the remote_host_headers.py
file. However, you can still set those values in the System settings of the Configure Tower user interface.
Tower determines the remote host’s IP address by searching through the list of headers in REMOTE_HOST_HEADERS
until the FIRST IP address is located.
Note
Header names are constructed using the following logic:
With the exception of CONTENT_LENGTH
and CONTENT_TYPE
, any HTTP headers in the request are converted to META keys by converting all characters to uppercase, replacing any hyphens with underscores, and adding an HTTP_
prefix to the name. For example, a header called X-Barkley
would be mapped to the META key HTTP_X_Barkley
.
For more information on HTTP request and response objects, refer to: https://docs.djangoproject.com/en/1.8/ref/request-response/#django.http.HttpRequest.META
Note
If using SSL termination at the load balancer and forwarding traffic to a different port on the tower node (443 -> 80), set the following values in the /etc/tower/conf.d/custom.py
file accordingly:
USE_X_FORWARDED_PORT = True
USE_X_FORWARDED_HOST = True
When Tower is configured with REMOTE_HOST_HEADERS = ['HTTP_X_FORWARDED_FOR', 'REMOTE_ADDR', 'REMOTE_HOST']
, it assumes that the value of X-Forwarded-For
has originated from the proxy/load balancer sitting in front of Tower. In a scenario where Tower is still reachable without use of the proxy/load balancer or when the proxy does not validate the header, X-Forwarded-For
can be spoofed fairly easily to fake the originating IP addresses. Using HTTP_X_FORWARDED_FOR
in the REMOTE_HOST_HEADERS
setting poses a vulnerability that essentially gives users access to certain resources that they should not have.
To avoid this, you can configure a list of “known proxies” that are allowed, which is the PROXY_IP_WHITELIST
setting via the settings API. Load balancers and hosts that are not on the list will result in a rejected request.
PROXY_IP_WHITELIST
only works if the proxies in the list are properly sanitizing header input and correctly setting an X-Forwarded-For
value equal to the real source IP of the client; the crux of this setting is that Tower can rely on the IPs/hostnames in PROXY_IP_WHITELIST
to provide non-spoofed values for the X-Forwarded-For
field.
HTTP_X_FORWARDED_FOR
should never be configured as an item in REMOTE_HOST_HEADERS
unless all of the following are satisfied:
You are using a proxied environment w/ ssl termination
The proxy provides sanitization/validation of the
X-Forwarded-For
header to prevent client spoofing
/etc/tower/conf.d/remote_host_headers.py
definesPROXY_IP_WHITELIST
that contains only the originating IP of trusted proxies/load balancers.
Note
If you do not need all of the traffic to be put through the proxy, then you can specify the IP scheme(s) that you want to exclude in the no_proxy
field. The list can be IP ranges or individual IPs, separated by a comma. This example shows a specified range of IPs in JSON format:
"https_proxy": "example.proxy.com:8080",
"http_proxy": "example.proxy.com:8080",
"no_proxy": "10.0.0.0/8"
Also, an SCM update with a no_proxy
configuration and a CIDR notation may not work for a particular SCM. The support for http_proxy
and no_proxy
depends on the the implementation in the application (git|hg|svn). For example, git does not support CIDR notation for no_proxy
because git is limited by its C library: https://curl.haxx.se/libcurl/c/CURLOPT_NOPROXY.html.
If you are behind a reverse proxy, you may want to setup a header field for HTTP_X_FORWARDED_FOR
. The X-Forwarded-For
(XFF) HTTP header field identifies the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.
REMOTE_HOST_HEADERS = ['HTTP_X_FORWARDED_FOR', 'REMOTE_ADDR', 'REMOTE_HOST']
A key configuration that you should consider is websocket setup in order to align the websocket config with your nginx / load balancer configuration.
Tower nodes connect to all other Tower nodes via websockets. This interconnect is used to distribute all websocket emitted messages to all other Tower nodes. This is required because any browser client websocket can subscribe to any job that may be running on any Tower node. In other words, websocket clients are not routed to specific Tower nodes. Any Tower node can handle any websocket request. Therefore, each Tower node must know about all websocket messages destined for all clients.
Tower will automatically handle discovery of other Tower nodes via the Instance record in the database. Tower must be told the port, protocol, and whether or not to verify certificates when establishing the websocket connections. This is configured using the following three settings:
BROADCAST_WEBSOCKET_PROTOCOL = 'http'
BROADCAST_WEBSOCKET_PORT = 80
BROADCAST_WEBSOCKET_VERIFY_CERT = False
For example, topologies that do SSL termination in front of Tower, at the load balancer level would have the following settings:
BROADCAST_WEBSOCKET_PROTOCOL = 'http'
BROADCAST_WEBSOCKET_PORT = 80
Topologies that perform SSL at each individual Tower node via nginx would have the following settings:
BROADCAST_WEBSOCKET_PROTOCOL = 'https'
BROADCAST_WEBSOCKET_PORT = 443
BROADCAST_WEBSOCKET_VERIFY_CERT = True
Note
It is intended that your nodes are broadcasting websocket traffic across a private, trusted subnet (and not the open Internet). Therefore, if you turn off HTTPS for websocket broadcasting, the websocket traffic (which is comprised mostly of Ansible playbook stdout), is sent between Tower nodes unencrypted.
PCC-IT International, Division of Power Capital Management, Inc. © 2024 All rights reserved.