scrapy 爬网站用代理的时候报错如下

2019-01-04 16:26:57 [csrc][scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.csrc.gov.cn/pub/newsite/xxpl/yxpl/index.html> (failed 1 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]
1131 2019-01-04 16:27:04 [csrc][scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.csrc.gov.cn/pub/newsite/xxpl/yxpl/index_1.html> (failed 1 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]
1132 2019-01-04 16:27:09 [csrc][scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.csrc.gov.cn/pub/newsite/xxpl/yxpl/index_2.html> (failed 1 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]
1133 2019-01-04 16:27:16 [csrc][scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.csrc.gov.cn/pub/newsite/xxpl/yxpl/index_3.html> (failed 1 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]
1134 2019-01-04 16:27:21 [csrc][scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.csrc.gov.cn/pub/newsite/xxpl/yxpl/index_4.html> (failed 1 times): [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]

查了国外网站，没找到原因是啥，其他网站没问题，就这个网站，报错不知道为啥？？

http://www.csrc.gov.cn/pub/newsite/xxpl/yxpl/index.html 这个网站

retrying

csrc

2019-01-04

debug

6 条回复 • 2019-01-05 10:54:42 +08:00

houzhimeng

2019-01-04 16:56:20 +08:00

https://stackoverflow.com/questions/50840101/curl-35-error1408f10bssl-routinesssl3-get-recordwrong-version-number

Ewig

2019-01-04 17:13:58 +08:00

@houzhimeng import base64

class proxy_middleware(object):

def __init__(self):
proxy_host = "w.t.16yn"
proxy_port = "***"
self.username = "***"
self.password = "**"
self.proxies = {"http": "http://{}:{}/".format(proxy_host, proxy_port)}
self.proxy_server = 'https://w5.t.16yun.cn:6469'
self.proxy_authorization = 'Basic ' + base64.urlsafe_b64encode(
bytes((self.username + ':' + self.password), 'ascii')).decode('utf8')

def process_request(self, request, spider):
request.meta['proxy'] = self.proxy_server
request.headers['Proxy-Authorization'] = self.proxy_authorization

我改成这样还是不行

15399905591

2019-01-04 17:28:04 +08:00

self.proxy_server = 'https://w5.t.16yun.cn:6469'
改成
self.proxy_server = 'http://w5.t.16yun.cn:6469'

huaerxiela

2019-01-04 17:39:44 +08:00

https://github.com/requests/requests/issues/2022

houzhimeng

2019-01-04 18:29:05 +08:00

https://github.com/scrapy/scrapy/issues/1855 你看看这个情况跟你一样么？

Ewig

2019-01-05 10:54:42 +08:00

@15399905591 为啥这个原因

scrapy 爬网站 用代理的时候 报错如下

scrapy 爬网站用代理的时候报错如下