Three ways to work with LWP via socks

LWP::Protocol::socks

This module allows to specify socks for LWP in the same way as you could specify http proxy, which LWP supports out-of-the-box. The problem of this module is that it is only supports socks version 5. But as we can see 4 version is still in use in the real world. To fix this problem I posted a patch on the cpan. Make sure that IO::Socket::Socks 0.2 or higher installed, because only in this version I added support for 4 version of the protocol. May be in the next few years author will accept the patch, but now if you want to use socks 4 you should patch the module yourself.

Here is a simple example of usage:

use LWP;
# You shouldn't say `use LWP::Protocol::socks'
# However you could do it to make sure it is installed
# If it is installed then it will be loaded automatically when needed
# But if it is not then the request will fail

# create lwp object as usual
my $lwp = LWP::UserAgent->new();
# and set socks 5 proxy with host=localhost and port=1080
$lwp->proxy(['http', 'https'], 'socks://localhost:1080');
# use 'socks://login:password@localhost:1080' instead if you
# need authentication
# ...
# thus you could set socks 4 proxy
# if module supports it
$lwp->proxy(['http', 'https'], 'socks4://localhost:1080');
# use 'socks://userid:@localhost:1080' instead if you need
# to specify user id for IDENT

Benefits of using LWP::Protocol::socks

  • Easy and logical proxy configuration, in the same way as you usual do it with LWP
  • Ability to setup different proxy for different objects
  • Authentication with login and password support
  • HTTPS support
  • Stable and supported IO::Socket::Socks module as backend

Cons of using LWP::Protocol::socks

  • No socks4 support out-of-the-box

IO::Socket::Socks::Wrapper

In fact this module allows to get worked via socks any pure perl module. To be more accurate - any connection initiated by perl connect() function. As it uses IO::Socket::Socks as backend it supports both 4 and 5 socks version. This module could be used to wrap any network connection or connections from separate modules. Because we talking about LWP let's wrap it up:

use IO::Socket::Socks::Wrapper (
	# wrap classes used by LWP for http and https connections
	Net::HTTP => {
		ProxyAddr => 'localhost',
		ProxyPort => 1080,
		SocksVersion => 4
	},
	Net::HTTPS => { # works since IO::Socket::Socks::Wrapper 0.02
		ProxyAddr => 'localhost',
		ProxyPort => 1080,
		SocksVersion => 4
	}
);
use LWP;

# then use LWP as usual
my $lwp = LWP::UserAgent->new();
# all connections will be done via specified socks proxy
my $resp = $lwp->get('http://www.google.com/');
print $resp->status_line, "\n";

$resp = $lwp->get('https://encrypted.google.com/');
print $resp->status_line, "\n"; 

Benefits of using IO::Socket::Socks::Wrapper

  • Universal solution: wrap up any module in the same way
  • It supports any options supported by IO::Socket::Socks

Cons of using IO::Socket::Socks::Wrapper

  • Global wrapping: you can't specify different proxy for different LWP objects

LWP::Protocol::http::SocksChain

This is extension to LWP::Protocol. It uses Net::SC as backend. From the documentation: Net::SC - perl module for create the chain from the SOCKS/HTTP proxies. So, this extension to the LWP::Protocol also could create connections via socks chain. It supports 4 and 5 socks versions. Let's see an example:

use LWP::Protocol::http::SocksChain;
use LWP::Protocol::https::SocksChain;
use LWP;

# turn on proxy chain
LWP::Protocol::implementor(http => 'LWP::Protocol::http::SocksChain');
LWP::Protocol::implementor(https => 'LWP::Protocol::https::SocksChain');

# setup proxy chain
@LWP::Protocol::http::SocksChain::EXTRA_SOCK_OPTS = (
	Chain_Len       => 1,
	Debug           => 0,
	# format: host:port:uid:pswd:socks_ver
	Chain_File_Data => ['localhost:1080:::5'],
);
@LWP::Protocol::https::SocksChain::EXTRA_SOCK_OPTS = (
	Chain_Len       => 1,
	Debug           => 0,
	Chain_File_Data => ['localhost:1080:::5'],
);

# then use LWP as usual
my $lwp = LWP::UserAgent->new();
# all connections will be done via specified socks proxy
# ...

Benefits of using LWP::Protocol::http::SocksChain

  • Allows to create connections via socks chain

Cons of using LWP::Protocol::http::SocksChain

  • Looks like ugly hack


Conclusion:

All of this three modules have roughly equal opportunities. All three supports socks 5 and 4 version (first with a patch) and authorization with login and password. The first two uses IO::Socket::Socks as backend, third uses own backend, called Net::SC.

LWP::Protocol::socks is the most simple and logical way to include socks support in the LWP. IO::Socket::Socks::Wrapper is the most universal solution if you need to include socks support everywhere. LWP::Protocol::http::SocksChain is unique solution for creating connections via socks chains.

Read original modules documentation to get more information.

1 comments:

Anonymous said...

Crap, last way doesn't works with TOR proxy.
I cannot to force it works well, seems like somthing wrong with socks or anything else..
Always get:
HTTP/1.1 200 OK
Connection: close
Date: Thu, 10 Jan 2013 15:03:46 GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Client-Aborted: die
Client-Date: Thu, 10 Jan 2013 15:03:55 GMT
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance CA-3
Client-SSL-Cert-Subject: /C=US/ST=Massachusetts/L=Walpole/O=The Tor Project, Inc./CN=*.torproject.org
Client-SSL-Cipher: DHE-RSA-AES256-SHA
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked
Strict-Transport-Security: max-age=15768000
X-Died: Can't read entity body: Resource temporarily unavailable at ...

Post a Comment