Questions tagged [python-requests]

USE ONLY FOR THE PYTHON REQUESTS LIBRARY. Requests is a full-featured Python HTTP library with an easy-to-use, logical API.

Filter by
Sorted by
Tagged with
1054 votes
10 answers

How to POST JSON data with Python Requests?

I need to POST a JSON from a client to a server. I'm using Python 2.7.1 and simplejson. The client is using Requests. The server is CherryPy. I can GET a hard-coded JSON from the server (code not ...
Charles R's user avatar
  • 18k
1011 votes
11 answers

What are the differences between the urllib, urllib2, urllib3 and requests module?

In Python, what are the differences between the urllib, urllib2, urllib3 and requests modules? Why are there three? They seem to do the same thing...
Paul Biggar's user avatar
  • 27.7k
911 votes
34 answers

ImportError: No module named requests

I tried importing requests: import requests But I get an error: ImportError: No module named requests
user2476540's user avatar
  • 9,285
719 votes
4 answers

Correct way to try/except using Python requests module?

try: r = requests.get(url, params={'s': thing}) except requests.ConnectionError, e: print(e) Is this correct? Is there a better way to structure this? Will this cover all my bases?
John Smith's user avatar
  • 11.7k
603 votes
9 answers

Download large file in python with requests

Requests is a really nice library. I'd like to use it for downloading big files (>1GB). The problem is it's not possible to keep whole file in memory; I need to read it in chunks. And this is a ...
Roman Podlinov's user avatar
517 votes
29 answers

Python Requests throwing SSLError

I'm working on a simple script that involves CAS, jspring security check, redirection, etc. I would like to use Kenneth Reitz's python requests because it's a great piece of work! However, CAS ...
TedBurrows's user avatar
  • 5,451
504 votes
18 answers

How to download image using requests

I'm trying to download and save an image from the web using python's requests module. Here is the (working) code I used: img = urllib2.urlopen(settings.STATICMAP_URL.format(**data)) with open(path, '...
shkschneider's user avatar
  • 17.8k
475 votes
12 answers

How do I disable the security certificate check in Python requests

I am using import requests'https://foo.example', data={'bar':'baz'}) but I get a request.exceptions.SSLError. The website has an expired certficate, but I am not sending sensitive ...
Paul Draper's user avatar
  • 78.9k
441 votes
13 answers

How do I disable log messages from the Requests library?

By default, the Requests python library writes log messages to the console, along the lines of: Starting new HTTP connection (1): "GET / HTTP/1.1" 200 606 I'm ...
aknuds1's user avatar
  • 65.8k
401 votes
3 answers

What's the best way to parse a JSON response from the requests library?

I'm using the python requests module to send a RESTful GET to a server, for which I get a response in JSON. The JSON response is basically just a list of lists. What's the best way to coerce the ...
felix001's user avatar
  • 15.5k
393 votes
9 answers

How can I see the entire HTTP request that's being sent by my Python application?

In my case, I'm using the requests library to call PayPal's API over HTTPS. Unfortunately, I'm getting an error from PayPal, and PayPal support cannot figure out what the error is or what's causing it....
Chris B.'s user avatar
  • 85.9k
373 votes
14 answers

How to send a "multipart/form-data" with requests in python?

How to send a multipart/form-data with requests in python? How to send a file, I understand, but how to send the form data by this method can not understand.
agrynchuk's user avatar
  • 4,647
368 votes
5 answers

Python Requests - No connection adapters

I'm using the Requests: HTTP for Humans library and I got this weird error and I don't know what is mean. No connection adapters were found for '' Anybody has an idea?
Azd325's user avatar
  • 5,782
345 votes
9 answers

Python requests - print entire http request (raw)?

While using the requests module, is there any way to print the raw HTTP request? I don't want just the headers, I want the request line, headers, and content printout. Is it possible to see what ...
huggie's user avatar
  • 17.6k
339 votes
3 answers

Sending "User-agent" using Requests library in Python

I want to send a value for "User-agent" while requesting a webpage using Python Requests. I am not sure is if it is okay to send this as a part of the header, as in the code below: debug = {'verbose'...
user avatar
337 votes
6 answers

Can I set max_retries for requests.request?

The Python requests module is simple and elegant but one thing bugs me. It is possible to get a requests.exception.ConnectionError with a message like: Max retries exceeded with url: ... This ...
Kirill Zaitsev's user avatar
323 votes
22 answers

Timeout for python requests.get entire response

I'm gathering statistics on a list of websites and I'm using requests for it for simplicity. Here is my code: data=[] websites=['', ''] for w in websites: r= ...
Kiarash's user avatar
  • 7,398
322 votes
4 answers

Using headers with the Python requests library's get method

So I recently stumbled upon this great library for handling HTTP requests in Python; found here I love working with it, but I can't figure out ...
Breedly's user avatar
  • 13k
306 votes
13 answers

How to install packages offline?

What's the best way to download a python package and its dependencies from pypi for offline installation on another machine? Is there any easy way to do this with pip or easy_install? I'm trying to ...
Chris Drantz's user avatar
  • 3,063
297 votes
19 answers

Max retries exceeded with URL in requests

I'm trying to get the content of App Store > Business: import requests from lxml import html page = requests.get("") tree = html.fromstring(...
user3446000's user avatar
  • 3,015
264 votes
12 answers

Proxies with Python 'Requests' module

Just a short, simple one about the excellent Requests module for Python. I can't seem to find in the documentation what the variable 'proxies' should contain. When I send it a dict with a standard &...
user avatar
257 votes
3 answers

Python Request Post with param data

This is the raw request for an API call: POST HTTP/1.1 Accept-Encoding: gzip,deflate ...
slysid's user avatar
  • 5,246
247 votes
9 answers

How to upload file with python requests?

I'm performing a simple task of uploading a file using Python requests library. I searched Stack Overflow and no one seemed to have the same problem, namely, that the file is not received by the ...
scichris's user avatar
  • 2,627
237 votes
16 answers

SSL InsecurePlatform error when using Requests package

Im using Python 2.7.3 and Requests. I installed Requests via pip. I believe it's the latest version. I'm running on Debian Wheezy. I've used Requests lots of times in the past and never faced this ...
Luke Peckham's user avatar
  • 2,375
236 votes
2 answers

Requests -- how to tell if you're getting a 404

I'm using the Requests library and accessing a website to gather data from it with the following code: r = requests.get(url) I want to add error testing for when an improper URL is entered and a ...
user1427661's user avatar
  • 11.2k
225 votes
13 answers

Asynchronous Requests with Python requests

I tried the sample provided within the documentation of the requests library for python. With, I get the response codes, but I want to get the content of each page requested. This, for ...
trbck's user avatar
  • 5,217
200 votes
7 answers

How could I use requests in asyncio?

I want to do parallel http request tasks in asyncio, but I find that python-requests would block the event loop of asyncio. I've found aiohttp but it couldn't provide the service of http request using ...
flyer's user avatar
  • 9,310
195 votes
2 answers

What is the difference between 'content' and 'text'

I am using the terrific Python Requests library. I notice that the fine documentation has many examples of how to do something without explaining the why. For instance, both r.text and r.content are ...
dotancohen's user avatar
  • 30.2k
186 votes
3 answers

How to send cookies in a post request with the Python Requests library?

I'm trying to use the Requests library to send cookies with a post request, but I'm not sure how to actually set up the cookies based on its documentation. The script is for use on Wikipedia, and the ...
Ricardo Altamirano's user avatar
185 votes
9 answers

How to use Python requests to fake a browser visit a.k.a and generate User Agent? [duplicate]

I want to get the content from this website. If I use a browser like Firefox or Chrome I could get the real website page I want, but if I use the Python requests package (or wget command) to get it, ...
user1726366's user avatar
  • 2,256
181 votes
2 answers

Adding headers to requests module

Earlier I used httplib module to add a header in the request. Now I am trying the same thing with the requests module. This is the python request module I am using:
discky's user avatar
  • 14.9k
177 votes
11 answers

How to get Python requests to trust a self signed SSL certificate?

import requests data = {'foo':'bar'} url = '' r =, data=data) If the URL uses a self signed certificate, this fails with requests.exceptions.SSLError: [Errno 1] ...
Matthew Moisen's user avatar
176 votes
1 answer

zsh: no matches found: requests[security]

I am trying to run a python urllib2 script and getting this error: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and ...
Kristen's user avatar
  • 1,771
175 votes
8 answers

Python Requests and persistent sessions

I am using the requests module. I have figured out how to submit data to a login form on a website and retrieve the session key, but I can't see an obvious way to use this session key in subsequent ...
ChrisGuest's user avatar
  • 3,418
163 votes
6 answers

How do I read a response from Python Requests?

I have two Python scripts. One uses the Urllib2 library and one uses the Requests library. I have found Requests easier to implement, but I can't find an equivalent for urlib2's read() function. For ...
Oli's user avatar
  • 2,277
159 votes
3 answers

How to extract HTTP response body from a Python requests call?

I'm using the Python requests library. I'm trying to figure out how to extract the actual HTML body from a response. The code looks a bit like this: r = requests.get(...) print r.content This should ...
Stephen Gross's user avatar
158 votes
8 answers

Log all requests from the python-requests module

I am using python Requests. I need to debug some OAuth activity, and for that I would like it to log all requests being performed. I could get this information with ngrep, but unfortunately it is not ...
blueFast's user avatar
  • 41.5k
154 votes
6 answers

How to "log in" to a website using Python's Requests module?

I am trying to post a request to log in to a website using the Requests module in Python but its not really working. I'm new to I can't figure out if I should make my Username and Password ...
Marcus Johnson's user avatar
154 votes
9 answers

Python requests library how to pass Authorization header with single token

I have a request URI and a token. If I use: curl -s "<MY_URI>" -H "Authorization: TOK:<MY_TOKEN>" etc., I get a 200 and view the corresponding JSON data. So, I installed requests and ...
user avatar
144 votes
2 answers

Python Requests package: Handling xml response

I like very much the requests package and its comfortable way to handle JSON responses. Unfortunately, I did not understand if I can also process XML responses. Has anybody experience how to handle ...
Andy's user avatar
  • 9,503
143 votes
3 answers

How to fix "403 Forbidden" errors when calling APIs using Python requests?

I needed to parse a site, but I got a 403 Forbidden error. Here is the code: url = '' result = requests.get(url) print(result.content.decode()) The output is: <html> &...
Толкачёв Иван's user avatar
143 votes
3 answers

Difference between data and json parameters in Python Requests package

What is the difference between the data and json parameters in the Python Requests package? It is unclear from the documentation. Does this code: import requests import json d = {'a': 1} response = ...
user1507844's user avatar
  • 6,023
141 votes
8 answers

Python Requests library redirect new url

I've been looking through the Python Requests documentation but I cannot see any functionality for what I am trying to achieve. In my script I am setting allow_redirects=True. I would like to know ...
Daniel Pilch's user avatar
  • 2,097
139 votes
26 answers

Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."

I'm using requests in Python3 on Windows via Pycharm, and I'm receiving a "SSL module is not available" error. I've spent hours trying to figure out what could be causing this. I've ...
Frank Drin's user avatar
  • 1,633
137 votes
6 answers

Why doesn't requests.get() return? What is the default timeout that requests.get() uses?

In my script, requests.get never returns: import requests print ("requesting..") # This call never returns! r = requests.get( "http://www.some-site.example", proxies = {'...
Nawaz's user avatar
  • 354k
135 votes
6 answers

Download and save PDF file with Python requests module

I am trying to download a PDF file from a website and save it to disk. My attempts either fail with encoding errors or result in blank PDFs. In [1]: import requests In [2]: url = 'http://www.hrecos....
Jim's user avatar
  • 1,740
131 votes
6 answers

How do I use basic HTTP authentication with the Python Requests library?

I'm trying to use basic HTTP authentication in Python. I am using the Requests library: auth ='http://' + hostname, auth=HTTPBasicAuth(user, password)) request = requests.get('http://' +...
oleksii's user avatar
  • 1,387
124 votes
4 answers

How can I use cookies in Python Requests?

I am trying to log in to a page and access another link in the page. I get a "405 Not Allowed" error from this attempt: payload={'username'=<username>,'password'=<password>} with ...
user1474157's user avatar
  • 1,389
118 votes
9 answers

How to make python Requests work via SOCKS proxy

I'm using the great Requests library in my Python script: import requests r = requests.get("") print(r.text) I would like to use a SOCKS proxy, how can I do that? Requests ...
lithuak's user avatar
  • 6,038
118 votes
5 answers

Python Requests - How to use system ca-certificates (debian/ubuntu)?

I've installed a self-signed root ca cert into debian's /usr/share/ca-certificates/local and installed them with sudo dpkg-reconfigure ca-certificates. At this point true | gnutls-cli mysite.local is ...
ThorSummoner's user avatar
  • 16.8k

2 3 4 5