Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. I’m using Squid as a forward proxy for my crawler system. On my proxy server, I have a lot of IP addresses and I want all of requests going to outside will choose an IP address randomize.

acl src_01 random 1/04
acl src_02 random 1/03
acl src_03 random 1/02
acl src_04 random 1/01

tcp_outgoing_address 123.123.123.123 src_01
tcp_outgoing_address 123.123.123.124 src_02
tcp_outgoing_address 123.123.123.125 src_03
tcp_outgoing_address 123.123.123.126 src_04

Everything work exactly! But now, I’m crawling datas from Facebook, I have a lot of FB accounts and I want each account will use one IP address. How I configure Squid to do it?

My idea is using req_header, eg: client send a header fb-gr: gr1 will use IP 123.123.123.123

acl fb_gr1 req_header fb-gr ^gr1$
acl fb_gr2 req_header fb-gr ^gr2$
acl fb_gr3 req_header fb-gr ^gr3$
acl fb_gr4 req_header fb-gr ^gr4$

tcp_outgoing_address 123.123.123.123 fb_gr1
tcp_outgoing_address 123.123.123.124 fb_gr2
tcp_outgoing_address 123.123.123.125 fb_gr3
tcp_outgoing_address 123.123.123.126 fb_gr4

Below is full example Squid configuration

#
# Recommended minimum configuration:
#

## ACL based on request header
## Used for proxy request to facebook
acl fb_gr1 req_header fb-gr ^gr1$
acl fb_gr2 req_header fb-gr ^gr2$
acl fb_gr3 req_header fb-gr ^gr3$
acl fb_gr4 req_header fb-gr ^gr4$

tcp_outgoing_address 123.123.123.123 fb_gr1
tcp_outgoing_address 123.123.123.124 fb_gr2
tcp_outgoing_address 123.123.123.125 fb_gr3
tcp_outgoing_address 123.123.123.126 fb_gr4

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl acl1 src 192.168.2.2

acl SSL_ports port 443
acl Safe_ports port 80        # http
acl Safe_ports port 21        # ftp
acl Safe_ports port 443        # https
acl Safe_ports port 70        # gopher
acl Safe_ports port 210        # wais
acl Safe_ports port 1025-65535    # unregistered ports
acl Safe_ports port 280        # http-mgmt
acl Safe_ports port 488        # gss-http
acl Safe_ports port 591        # filemaker
acl Safe_ports port 777        # multiling http
acl CONNECT method CONNECT
acl HEAD method HEAD # allow HEAD method

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow cloudntl manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow acl1

# Allow HEAD method
http_access allow HEAD

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 10.1.2.3:3128

acl src_01 random 1/04
acl src_02 random 1/03
acl src_03 random 1/02
acl src_04 random 1/01

tcp_outgoing_address 123.123.123.123 src_01
tcp_outgoing_address 123.123.123.124 src_02
tcp_outgoing_address 123.123.123.125 src_03
tcp_outgoing_address 123.123.123.126 src_04

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:        1440    20%    10080
refresh_pattern ^gopher:    1440    0%    1440
refresh_pattern -i (/cgi-bin/|\?) 0    0%    0
refresh_pattern .        0    20%    4320

# Disable cache, proxy only
cache deny all

request_header_replace User-Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36

via off
forwarded_for off
follow_x_forwarded_for deny all
request_header_access X-Forwarded-For deny all
reply_header_access X-Cache deny all
reply_header_access X-Cache-Lookup deny all

logformat combined %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
access_log daemon:/var/log/squid/access.log combined