High-performance, event-driven web server, reverse proxy, and load balancer for HTTP, TCP, and UDP traffic.

Addresses below are RFC 5737 documentation ranges or placeholders - swap in your own.

Table of Contents#

  1. Overview
  2. Basic Configuration
  3. Upstream Directive
  4. Map Feature
  5. Reverse Proxy and WebSocket
  6. Load Balancing
  7. TCP/UDP Proxy (Stream)
  8. Authentication
  9. Rate Limiting
  10. Proxy Caching
  11. FastCGI Configuration
  12. gRPC Proxying
  13. Performance Tuning
  14. Troubleshooting
  15. See Also
  16. Sources

1. Overview#

Nginx (pronounced "engine-x") is an asynchronous, event-driven web server designed for high concurrency and low resource usage. Originally built to solve the C10K problem, it handles thousands of simultaneous connections within a single worker process without creating threads per connection. Nginx is widely used as a reverse proxy, load balancer, TLS terminator, HTTP cache, and static file server.

Key capabilities:

  • Event-driven architecture with non-blocking I/O
  • HTTP/2 and HTTP/3 (QUIC) support
  • TLS termination with OCSP stapling
  • Layer 4 (TCP/UDP) and Layer 7 (HTTP) proxying
  • Built-in caching, compression, and rate limiting
  • Modular design with dynamic module loading

2. Basic Configuration#

2.1 Configuration File#

The main configuration file is typically /etc/nginx/nginx.conf. This example covers performance, security, SSL, and compression:

user http;
worker_processes auto;
worker_cpu_affinity auto;

events {
    multi_accept on;
    worker_connections 1024;
}

http {
    charset utf-8;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    server_tokens off;
    log_not_found off;
    types_hash_max_size 4096;

    # SSL
    ssl_prefer_server_ciphers on;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_dhparam /etc/nginx/dhparam.pem;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM";
    ssl_ecdh_curve secp384r1;
    ssl_stapling on;
    ssl_stapling_verify on;

    # Security Headers
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";
    # add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";

    # MIME
    include mime.types;
    default_type application/octet-stream;

    # Logging
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log warn;

    # Compression
    gzip on;
    gzip_proxied any;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript
               text/xml application/xml application/xml+rss text/javascript;
    gzip_comp_level 5;
    gzip_buffers 16 8k;

    # Load additional configs
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.conf;

    # Default: redirect HTTP to HTTPS
    server {
        listen 80 default_server;
        server_name _;
        return 301 https://$host$request_uri;
    }

    # Default: reject unknown HTTPS hosts
    server {
        listen 443 ssl;
        server_name _;
        ssl_certificate self-signed.crt;
        ssl_certificate_key self-signed.key;
        return 444;
    }
}

Generate the DH parameter file and self-signed certificate:

openssl dhparam -out /etc/nginx/dhparam.pem 4096
openssl genrsa -out /etc/nginx/self-signed.key 2048
openssl req -new -x509 -sha256 -key /etc/nginx/self-signed.key \
    -out /etc/nginx/self-signed.crt -days 3650 -subj "/CN=localhost"

3. Upstream Directive#

3.1 Purpose and Syntax#

The upstream directive defines a group of backend servers for load balancing and failover. It is used within the http context and referenced by proxy_pass.

upstream <name> {
    server <address> [parameters];
    ...
}

Parameters:

  • weight=<n> - Server weight for weighted balancing (default 1)
  • max_fails=<n> - Failures before marking server unavailable (default 1)
  • fail_timeout=<time> - Duration to consider server unavailable after max_fails (default 10s)
  • backup - Only used when all primary servers are unavailable
  • down - Marks server as permanently offline
  • max_conns=<n> - Maximum concurrent connections to the server

3.2 Example#

upstream backend {
    server backend1.example.com weight=3;
    server backend2.example.com weight=2;
    server backend3.example.com backup;
}

4. Map Feature#

The map directive creates variables whose values depend on other variables. Defined in the http context, evaluated lazily on first use.

4.1 Syntax#

map <source_variable> <destination_variable> {
    default <default_value>;
    <pattern> <result_value>;
    ...
}
  • Patterns can be literal strings or regular expressions (prefixed with ~ or ~* for case-insensitive)
  • default sets the value when no pattern matches

4.2 Example: Browser Detection#

map $http_user_agent $browser {
    default "Unknown";
    ~*chrome "Chrome";
    ~*firefox "Firefox";
    ~*safari "Safari";
}

4.3 Example: WebSocket Upgrade#

map $http_upgrade $connection_upgrade {
    default upgrade;
    ""      close;
}

5. Reverse Proxy and WebSocket#

5.1 Handling WebSocket Connections#

WebSocket requires the Upgrade and Connection headers to be forwarded. The map directive from Section 4.3 is commonly used to handle this conditionally.

5.2 Reverse Proxy Example#

A complete reverse proxy with WebSocket support, SSL, and proper header forwarding:

server {
    listen 443 ssl;
    server_name example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $server_name;

        # WebSocket support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        # Real client IP forwarding
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

6. Load Balancing#

6.1 Load Balancing Methods#

MethodDirectiveBehavior
Round Robin(default)Distributes requests evenly across servers
Least Connectionsleast_connRoutes to server with fewest active connections
IP Haship_hashRoutes based on client IP for session persistence
Generic Hashhash <key>Routes based on a user-defined key
RandomrandomSelects a random server, optionally weighted

6.2 Examples#

# Round Robin (default)
upstream round_robin {
    server server1.example.com;
    server server2.example.com;
}

# Least Connections
upstream least_connections {
    least_conn;
    server server1.example.com;
    server server2.example.com;
}

# IP Hash for session persistence
upstream ip_hash {
    ip_hash;
    server server1.example.com;
    server server2.example.com;
}

# Consistent hashing by URI (good for caching)
upstream uri_hash {
    hash $request_uri consistent;
    server server1.example.com;
    server server2.example.com;
}

7. TCP/UDP Proxy (Stream)#

The stream module handles Layer 4 proxying for non-HTTP protocols. It is defined outside the http block.

7.1 Basic TCP Proxy#

stream {
    upstream backend {
        server backend1.example.com:12345;
        server backend2.example.com:12345;
    }

    server {
        listen 12345;
        proxy_pass backend;
    }
}

7.2 SSL/TLS Termination (Stream)#

stream {
    upstream secure_backend {
        server backend1.example.com:12345;
        server backend2.example.com:12345;
    }

    server {
        listen 12345 ssl;
        ssl_certificate /etc/nginx/ssl/cert.pem;
        ssl_certificate_key /etc/nginx/ssl/key.pem;
        proxy_pass secure_backend;
    }
}

7.3 SNI-Based Routing#

Route encrypted traffic to different backends based on the TLS SNI hostname without terminating TLS:

stream {
    map $ssl_preread_server_name $backend {
        app1.example.com app1_backend;
        app2.example.com app2_backend;
        default          default_backend;
    }

    upstream app1_backend {
        server 192.0.2.11:443;
    }

    upstream app2_backend {
        server 192.0.2.12:443;
    }

    upstream default_backend {
        server 192.0.2.10:443;
    }

    server {
        listen 443;
        proxy_pass $backend;
        ssl_preread on;
    }
}

8. Authentication#

8.1 Basic Authentication#

Create a password file and configure the location:

# Install htpasswd utility (apache2-utils on Debian, httpd-tools on RHEL)
htpasswd -c /etc/nginx/.htpasswd <username>
location /admin {
    auth_basic "Restricted Area";
    auth_basic_user_file /etc/nginx/.htpasswd;
    proxy_pass http://backend;
}

8.2 Subrequest Authentication (OAuth/External Auth)#

Delegate authentication to an external service. Nginx sends a subrequest and grants access only if the auth service returns 2xx:

location /protected {
    auth_request /auth;
    auth_request_set $auth_user $upstream_http_x_auth_user;
    proxy_set_header X-Auth-User $auth_user;
    proxy_pass http://backend;
}

location = /auth {
    internal;
    proxy_pass http://auth-service:8080/validate;
    proxy_pass_request_body off;
    proxy_set_header Content-Length "";
    proxy_set_header X-Original-URI $request_uri;
    proxy_set_header X-Original-Method $request_method;
}

This pattern works with OAuth2 Proxy, Authelia, Authentik, Vouch, and similar authentication gateways.

8.3 IP-Based Access Control#

location /internal {
    allow 10.0.0.0/8;
    allow 192.168.0.0/16;
    deny all;
    proxy_pass http://backend;
}

Combine with satisfy any to allow access if either IP or auth succeeds:

location /admin {
    satisfy any;
    allow 10.0.0.0/8;
    deny all;
    auth_basic "Admin";
    auth_basic_user_file /etc/nginx/.htpasswd;
    proxy_pass http://backend;
}

9. Rate Limiting#

9.1 Request Rate Limiting#

Define a shared memory zone and apply limits:

http {
    # 10 MB zone, keyed by client IP, 10 requests/second
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

    server {
        location /api/ {
            # Allow burst of 20, delay processing beyond 10
            limit_req zone=api_limit burst=20 delay=10;
            limit_req_status 429;
            proxy_pass http://backend;
        }
    }
}

9.2 Connection Limiting#

http {
    limit_conn_zone $binary_remote_addr zone=conn_limit:10m;

    server {
        location /downloads/ {
            limit_conn conn_limit 5;       # Max 5 concurrent connections per IP
            limit_rate 500k;               # Throttle to 500 KB/s per connection
            limit_rate_after 10m;          # Throttle only after first 10 MB
            proxy_pass http://backend;
        }
    }
}

9.3 Whitelist Certain IPs from Rate Limits#

geo $limit {
    default 1;
    10.0.0.0/8 0;
    192.168.0.0/16 0;
}

map $limit $limit_key {
    0 "";
    1 $binary_remote_addr;
}

limit_req_zone $limit_key zone=api_limit:10m rate=10r/s;

10. Proxy Caching#

10.1 Basic Cache Configuration#

http {
    proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=app_cache:10m
                     max_size=1g inactive=60m use_temp_path=off;

    server {
        location / {
            proxy_cache app_cache;
            proxy_cache_valid 200 302 10m;
            proxy_cache_valid 404 1m;
            proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
            proxy_cache_lock on;
            proxy_cache_min_uses 2;

            # Add cache status header for debugging
            add_header X-Cache-Status $upstream_cache_status;

            proxy_pass http://backend;
        }
    }
}

10.2 Cache Key Customization#

proxy_cache_key "$scheme$request_method$host$request_uri";

10.3 Bypass and Purge#

# Bypass cache for authenticated users
proxy_cache_bypass $http_authorization;
proxy_no_cache $http_authorization;

# Bypass via custom header
map $http_x_purge $purge_method {
    default 0;
    "1"     1;
}

location / {
    proxy_cache app_cache;
    proxy_cache_bypass $purge_method;
    proxy_pass http://backend;
}

10.4 Cache Status Values#

ValueMeaning
MISSResponse not in cache; fetched from backend
HITResponse served from cache
EXPIREDCache entry expired; refreshed from backend
STALEStale entry served (backend unreachable)
UPDATINGStale entry served while cache updates in background
REVALIDATEDCache entry revalidated with backend (304 response)
BYPASSCache was bypassed per configuration

11. FastCGI Configuration#

11.1 PHP-FPM (FastCGI)#

server {
    listen 443 ssl;
    server_name example.com;
    root /var/www/example.com;
    index index.php index.html;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/run/php/php-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;

        fastcgi_buffering on;
        fastcgi_buffer_size 16k;
        fastcgi_buffers 16 16k;
        fastcgi_connect_timeout 60s;
        fastcgi_send_timeout 60s;
        fastcgi_read_timeout 60s;
    }

    # Deny access to .ht files
    location ~ /\.ht {
        deny all;
    }
}

11.2 FastCGI Cache#

http {
    fastcgi_cache_path /var/cache/nginx/fastcgi levels=1:2
                       keys_zone=fcgi_cache:10m max_size=512m inactive=30m;

    server {
        location ~ \.php$ {
            fastcgi_pass unix:/run/php/php-fpm.sock;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            include fastcgi_params;

            fastcgi_cache fcgi_cache;
            fastcgi_cache_valid 200 10m;
            fastcgi_cache_use_stale error timeout;
            fastcgi_cache_key "$scheme$request_method$host$request_uri";

            add_header X-FastCGI-Cache $upstream_cache_status;
        }
    }
}

12. gRPC Proxying#

12.1 Basic gRPC Proxy#

Nginx can proxy gRPC traffic (HTTP/2 based). Use grpc_pass instead of proxy_pass:

server {
    listen 443 ssl http2;
    server_name grpc.example.com;

    ssl_certificate /etc/letsencrypt/live/grpc.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/grpc.example.com/privkey.pem;

    location / {
        grpc_pass grpc://127.0.0.1:50051;
        grpc_set_header X-Real-IP $remote_addr;
    }
}

12.2 gRPC with TLS to Backend#

location / {
    grpc_pass grpcs://127.0.0.1:50051;
    grpc_ssl_certificate /etc/nginx/client.pem;
    grpc_ssl_certificate_key /etc/nginx/client.key;
}

12.3 gRPC Load Balancing#

upstream grpc_servers {
    server 192.0.2.11:50051;
    server 192.0.2.12:50051;
}

server {
    listen 443 ssl http2;
    server_name grpc.example.com;

    ssl_certificate /etc/letsencrypt/live/grpc.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/grpc.example.com/privkey.pem;

    location / {
        grpc_pass grpc://grpc_servers;

        # Long-lived streaming RPCs
        grpc_read_timeout 1d;
        grpc_send_timeout 1d;
        grpc_socket_keepalive on;
    }
}

12.4 gRPC Timeout Configuration#

For long-lived gRPC streams (e.g., bidirectional streaming), increase timeouts:

grpc_read_timeout 24h;
grpc_send_timeout 24h;
grpc_socket_keepalive on;
keepalive_timeout 24h;

13. Performance Tuning#

13.1 Worker Processes and Connections#

# Match worker count to CPU cores
worker_processes auto;
worker_cpu_affinity auto;

# Increase file descriptor limit per worker
worker_rlimit_nofile 8192;

events {
    # Maximum connections per worker
    worker_connections 4096;

    # Accept multiple connections at once
    multi_accept on;

    # Use epoll on Linux for best performance
    use epoll;
}

13.2 Buffer Sizes#

Tune buffers to match your typical request and response sizes:

http {
    # Client request buffers
    client_body_buffer_size 16k;
    client_header_buffer_size 1k;
    large_client_header_buffers 4 16k;
    client_max_body_size 100m;

    # Proxy buffers (for reverse proxy responses)
    proxy_buffer_size 4k;
    proxy_buffers 8 16k;
    proxy_busy_buffers_size 32k;

    # Temp file threshold (write to disk if response exceeds buffers)
    proxy_max_temp_file_size 1024m;
}

13.3 Keepalive Connections#

http {
    # Client keepalive
    keepalive_timeout 65;
    keepalive_requests 1000;

    # Upstream keepalive (reuse connections to backends)
    upstream backend {
        server 192.0.2.11:8080;
        server 192.0.2.12:8080;
        keepalive 32;
        keepalive_requests 1000;
        keepalive_timeout 60s;
    }

    server {
        location / {
            proxy_pass http://backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
        }
    }
}

13.4 Static File Serving#

location /static/ {
    alias /var/www/static/;
    expires 30d;
    access_log off;
    add_header Cache-Control "public, immutable";

    # Open file cache
    open_file_cache max=1000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
}

13.5 System-Level Tuning#

# /etc/sysctl.d/nginx.conf
net.core.somaxconn = 65535
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15

Troubleshooting#

IssueCauseSolution
502 Bad GatewayBackend is down or not respondingCheck backend health; verify proxy_pass address and port
504 Gateway TimeoutBackend too slow to respondIncrease proxy_read_timeout; check backend performance
413 Request Entity Too LargeRequest body exceeds limitIncrease client_max_body_size
connect() failed - no live upstreamsAll upstream servers marked downCheck backend health; adjust max_fails and fail_timeout
WebSocket connection dropsMissing Upgrade/Connection headersAdd proxy_http_version 1.1 and Upgrade/Connection headers
High memory usageToo many worker connections or large buffersReduce worker_connections or buffer sizes; check for connection leaks
SSL: error:0A000086Certificate chain incomplete or wrong orderEnsure fullchain.pem includes intermediate certificates
Old workers consuming CPU after reloadGraceful shutdown workers processing long connectionskill old worker PIDs; check for long-lived WebSocket/gRPC connections
too many open filesFile descriptor limit too lowIncrease worker_rlimit_nofile and system nofile limit
gRPC stream disconnectsDefault timeouts too short for streamingSet grpc_read_timeout and grpc_send_timeout to match stream duration

See Also#

Sources#