X-Forwarded-For is an HTTP extension header. The HTTP/1.1 (RFC 2616) protocol does not define it. It was originally introduced by Squid, a caching proxy software, to represent the real IP of the HTTP requester. Today it has become a de facto standard, widely used by HTTP proxies, load balancers, and other forwarding services, and has been written into the RFC 7239 (Forwarded HTTP Extension) standard.

The X-Forwarded-For request header format is very simple, like this:

X-Forwarded-For: client, proxy1, proxy2

As you can see, the XFF content consists of multiple parts separated by "comma + space". The first one is the IP of the device farthest from the server, followed by the IP of each proxy device.

If an HTTP request passes through three proxies Proxy1, Proxy2, and Proxy3 before reaching the server, with IPs IP1, IP2, and IP3 respectively, and the user's real IP is IP0, then according to the XFF standard, the server will eventually receive the following information:

X-Forwarded-For: IP0, IP1, IP2

Proxy3 directly connects to the server. It appends IP2 to XFF, indicating that it is forwarding the request on behalf of Proxy2. IP3 is not in the list; IP3 can be obtained on the server side via the Remote Address field. We know HTTP connections are based on TCP connections. The HTTP protocol has no concept of IP. Remote Address comes from the TCP connection and represents the IP of the device that established the TCP connection with the server. In this example, it is IP3.

Remote Address cannot be forged because establishing a TCP connection requires a three-way handshake. If the source IP is forged, the TCP connection cannot be established, and there will be no subsequent HTTP request. Different languages obtain Remote Address in different ways. For example, PHP uses $_SERVER["REMOTE_ADDR"], Node.js uses req.connection.remoteAddress, but the principle is the same.

Problem

With the background knowledge above, let's start talking about the problem. I wrote a very simple Web Server in Node.js for testing. The HTTP protocol is language-agnostic. Node.js is used here only for convenience; any other language can reach the same conclusion. The same applies to Nginx in this article. If you are interested, Apache or other Web Servers would also work.

The following code listens on port 9009 and outputs some information after receiving an HTTP request:

var http = require('http');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.write('remoteAddress: ' + req.connection.remoteAddress + '\n');
    res.write('x-forwarded-for: ' + req.headers['x-forwarded-for'] + '\n');
    res.write('x-real-ip: ' + req.headers['x-real-ip'] + '\n');
    res.end();
}).listen(9009, '0.0.0.0');

In addition to the Remote Address and X-Forwarded-For introduced earlier, this code also has an X-Real-IP, which is another custom header field. X-Real-IP is often used by HTTP proxies to represent the IP of the device that established a TCP connection with it. This device may be another proxy or the actual requester. Note that X-Real-IP is not currently part of any standard. Proxies and Web applications can agree to use any custom header to pass this information.

Now you can directly access this Node.js service using the domain name + port number, and also configure an Nginx reverse proxy:

location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://127.0.0.1:9009/;
    proxy_redirect off;
}

My Nginx listens on port 80, so I can access the service forwarded by Nginx without a port.

Test direct access to the Node service:

curl http://t1.imququ.com:9009/

remoteAddress: 114.248.238.236
x-forwarded-for: undefined
x-real-ip: undefined

Since my computer directly connected to the Node.js service, the Remote Address is my IP. I did not specify any extra custom headers, so the latter two fields are both undefined.

Now access the service forwarded by Nginx:

curl http://t1.imququ.com/

remoteAddress: 127.0.0.1
x-forwarded-for: 114.248.238.236
x-real-ip: 114.248.238.236

This time, my computer accesses the Node.js service through Nginx, and the Remote Address obtained is actually Nginx's local IP. The two lines in the Nginx configuration earlier took effect, adding two custom headers to the request:

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

In fact, in production environments, Web applications are usually deployed in the second way above, which has many benefits. But this introduces a hidden danger: many Web applications obtain the real user IP by taking it from HTTP request headers.

HTTP request headers can be arbitrarily constructed. Let's use curl's -H parameter to construct X-Forwarded-For and X-Real-IP and test again.

Directly access the Node.js service:

curl http://t1.imququ.com:9009/ -H 'X-Forwarded-For: 1.1.1.1' -H 'X-Real-IP: 2.2.2.2'

remoteAddress: 114.248.238.236
x-forwarded-for: 1.1.1.1
x-real-ip: 2.2.2.2

For Web applications, X-Forwarded-For and X-Real-IP are just two ordinary request headers, so they are output as-is without any processing. This shows that for direct connection deployment, except for the Remote Address obtained from the TCP connection, IP information carried in request headers cannot be trusted.

Access the service forwarded by Nginx:

curl http://t1.imququ.com/ -H 'X-Forwarded-For: 1.1.1.1' -H 'X-Real-IP: 2.2.2.2'

remoteAddress: 127.0.0.1
x-forwarded-for: 1.1.1.1, 114.248.238.236
x-real-ip: 114.248.238.236

This time, Nginx appends my IP after X-Forwarded-For and overwrites the X-Real-IP request header with my IP. This shows that with Nginx's processing, the last section of X-Forwarded-For and the entire content of X-Real-IP cannot be forged and can be used to obtain the user IP.

User IP is often used in scenarios related to Web security, such as checking the user's login region and IP-based access frequency control. In such scenarios, ensuring that the IP cannot be forged is more important. Based on the previous tests and analysis, for Web applications directly deployed to users, you must use the Remote Address obtained from the TCP connection. For Web applications deployed behind a reverse proxy like Nginx, after correctly configuring the Set Header behavior, you can use the X-Real-IP or the last section of X-Forwarded-For passed by Nginx (in fact, they are equivalent).

So how does a Web application itself determine whether a request came directly or was forwarded by a controllable proxy? Adding an extra request header during proxy forwarding is one way, but it is not very reliable because request headers are too easy to forge. If you must use this approach, the custom header should be long enough and rare enough, and also be kept safe so it does not leak.

Checking whether Remote Address is a local IP is also a method, but it is not perfect, because accessing from the server where Nginx resides, whether direct or through the Nginx proxy, the Remote Address is always 127.0.0.1. This issue is usually negligible. More troublesome is that the reverse proxy server and the actual Web application may not be deployed on the same server. Therefore, a more reasonable approach is to maintain a list of all proxy server IPs. After the Web application gets the Remote Address, it can compare against the list one by one to determine how the request was made.

Usually, to simplify logic, production environments block direct access to Web applications via a port, and only allow access through Nginx. Does that mean there are no problems? Not necessarily.

First, if the user really accesses Nginx through a proxy, the last section of X-Forwarded-For and X-Real-IP will get the proxy's IP. Security-related scenarios can only use this, but some scenarios, such as showing the local weather based on the IP, need to obtain the user's real IP as much as possible. At this time, the first IP in X-Forwarded-For can come in handy. At this point, we need to pay attention to an issue. Let's use the previous example for testing:

curl http://t1.imququ.com/ -H 'X-Forwarded-For: unknown, <>"1.1.1.1'

remoteAddress: 127.0.0.1
x-forwarded-for: unknown, <>"1.1.1.1, 114.248.238.236
x-real-ip: 114.248.238.236

The last section of X-Forwarded-For is appended by Nginx, but the preceding parts all come from the request headers that Nginx received. This part is user input and completely untrustworthy. You need to be extra careful when using it. Only use it if it matches the IP format; otherwise, it can easily cause security vulnerabilities such as SQL injection or XSS.

Conclusion

1. Web applications that directly provide services to the outside world can only obtain the IP through Remote Address when performing security-related operations, and cannot trust any request headers;
2. Web applications using Nginx or other Web Servers for reverse proxy, with correct configuration, should useX-Forwarded-Forthe last section orX-Real-IPto obtain the IP (because Remote Address gets the internal IP of the server where Nginx is located); at the same time, direct external access to the Web application should be forbidden;
3. In scenarios not related to security, such as displaying the local weather by IP, you can get the IP fromX-Forwarded-Foran earlier position, but you need to validate the IP format.

PS: Some articles online suggest configuring Nginx like this, but it is actually not reasonable:

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;

With this configuration, security is indeed improved, but it also erases all proxy information before the request reaches Nginx, making it impossible to provide better services for users who actually use proxies. You should still understand the underlying principles and analyze specific scenarios on a case-by-case basis.

Original address: https://imququ.com/post/x-forwarded-for-header-in-http.html