Questions tagged [python-requests]
USE ONLY FOR THE PYTHON REQUESTS LIBRARY. Requests is a full-featured Python HTTP library with an easy-to-use, logical API.
21,790
questions
1054
votes
10
answers
1.7m
views
How to POST JSON data with Python Requests?
I need to POST a JSON from a client to a server. I'm using Python 2.7.1 and simplejson. The client is using Requests. The server is CherryPy. I can GET a hard-coded JSON from the server (code not ...
- 18k
1011
votes
11
answers
469k
views
What are the differences between the urllib, urllib2, urllib3 and requests module?
In Python, what are the differences between the urllib, urllib2, urllib3 and requests modules? Why are there three? They seem to do the same thing...
- 27.7k
911
votes
34
answers
2.5m
views
ImportError: No module named requests
I tried importing requests:
import requests
But I get an error:
ImportError: No module named requests
- 9,285
719
votes
4
answers
917k
views
Correct way to try/except using Python requests module?
try:
r = requests.get(url, params={'s': thing})
except requests.ConnectionError, e:
print(e)
Is this correct? Is there a better way to structure this? Will this cover all my bases?
- 11.7k
603
votes
9
answers
593k
views
Download large file in python with requests
Requests is a really nice library. I'd like to use it for downloading big files (>1GB).
The problem is it's not possible to keep whole file in memory; I need to read it in chunks. And this is a ...
- 23.8k
517
votes
29
answers
1.1m
views
Python Requests throwing SSLError
I'm working on a simple script that involves CAS, jspring security check, redirection, etc. I would like to use Kenneth Reitz's python requests because it's a great piece of work! However, CAS ...
- 5,451
504
votes
18
answers
634k
views
How to download image using requests
I'm trying to download and save an image from the web using python's requests module.
Here is the (working) code I used:
img = urllib2.urlopen(settings.STATICMAP_URL.format(**data))
with open(path, '...
- 17.8k
475
votes
12
answers
998k
views
How do I disable the security certificate check in Python requests
I am using
import requests
requests.post(url='https://foo.example', data={'bar':'baz'})
but I get a request.exceptions.SSLError.
The website has an expired certficate, but I am not sending sensitive ...
- 78.9k
441
votes
13
answers
203k
views
How do I disable log messages from the Requests library?
By default, the Requests python library writes log messages to the console, along the lines of:
Starting new HTTP connection (1): example.com
http://example.com:80 "GET / HTTP/1.1" 200 606
I'm ...
- 65.8k
401
votes
3
answers
821k
views
What's the best way to parse a JSON response from the requests library?
I'm using the python requests module to send a RESTful GET to a server, for which I get a response in JSON. The JSON response is basically just a list of lists.
What's the best way to coerce the ...
- 15.5k
393
votes
9
answers
329k
views
How can I see the entire HTTP request that's being sent by my Python application?
In my case, I'm using the requests library to call PayPal's API over HTTPS. Unfortunately, I'm getting an error from PayPal, and PayPal support cannot figure out what the error is or what's causing it....
- 85.9k
373
votes
14
answers
609k
views
How to send a "multipart/form-data" with requests in python?
How to send a multipart/form-data with requests in python? How to send a file, I understand, but how to send the form data by this method can not understand.
- 4,647
368
votes
5
answers
346k
views
Python Requests - No connection adapters
I'm using the Requests: HTTP for Humans library and I got this weird error and I don't know what is mean.
No connection adapters were found for '192.168.1.61:8080/api/call'
Anybody has an idea?
- 5,782
345
votes
9
answers
482k
views
Python requests - print entire http request (raw)?
While using the requests module, is there any way to print the raw HTTP request?
I don't want just the headers, I want the request line, headers, and content printout. Is it possible to see what ...
- 17.6k
339
votes
3
answers
491k
views
Sending "User-agent" using Requests library in Python
I want to send a value for "User-agent" while requesting a webpage using Python Requests. I am not sure is if it is okay to send this as a part of the header, as in the code below:
debug = {'verbose'...
user1289853
337
votes
6
answers
315k
views
Can I set max_retries for requests.request?
The Python requests module is simple and elegant but one thing bugs me.
It is possible to get a requests.exception.ConnectionError with a message like:
Max retries exceeded with url: ...
This ...
- 4,521
323
votes
22
answers
579k
views
Timeout for python requests.get entire response
I'm gathering statistics on a list of websites and I'm using requests for it for simplicity. Here is my code:
data=[]
websites=['http://google.com', 'http://bbc.co.uk']
for w in websites:
r= ...
- 7,398
322
votes
4
answers
728k
views
Using headers with the Python requests library's get method
So I recently stumbled upon this great library for handling HTTP requests in Python; found here http://docs.python-requests.org/en/latest/index.html.
I love working with it, but I can't figure out ...
- 13k
306
votes
13
answers
514k
views
How to install packages offline?
What's the best way to download a python package and its dependencies from pypi for offline installation on another machine? Is there any easy way to do this with pip or easy_install? I'm trying to ...
- 3,063
297
votes
19
answers
1.1m
views
Max retries exceeded with URL in requests
I'm trying to get the content of App Store > Business:
import requests
from lxml import html
page = requests.get("https://itunes.apple.com/in/genre/ios-business/id6000?mt=8")
tree = html.fromstring(...
- 3,015
264
votes
12
answers
599k
views
Proxies with Python 'Requests' module
Just a short, simple one about the excellent Requests module for Python.
I can't seem to find in the documentation what the variable 'proxies' should contain. When I send it a dict with a standard &...
user1064306
257
votes
3
answers
658k
views
Python Request Post with param data
This is the raw request for an API call:
POST http://192.168.3.45:8080/api/v2/event/log?sessionKey=b299d17b896417a7b18f46544d40adb734240cc2&format=json HTTP/1.1
Accept-Encoding: gzip,deflate
...
- 5,246
247
votes
9
answers
538k
views
How to upload file with python requests?
I'm performing a simple task of uploading a file using Python requests library. I searched Stack Overflow and no one seemed to have the same problem, namely, that the file is not received by the ...
- 2,627
237
votes
16
answers
202k
views
SSL InsecurePlatform error when using Requests package
Im using Python 2.7.3 and Requests. I installed Requests via pip. I believe it's the latest version. I'm running on Debian Wheezy.
I've used Requests lots of times in the past and never faced this ...
- 2,375
236
votes
2
answers
296k
views
Requests -- how to tell if you're getting a 404
I'm using the Requests library and accessing a website to gather data from it with the following code:
r = requests.get(url)
I want to add error testing for when an improper URL is entered and a ...
- 11.2k
225
votes
13
answers
529k
views
Asynchronous Requests with Python requests
I tried the sample provided within the documentation of the requests library for python.
With async.map(rs), I get the response codes, but I want to get the content of each page requested. This, for ...
- 5,217
200
votes
7
answers
197k
views
How could I use requests in asyncio?
I want to do parallel http request tasks in asyncio, but I find that python-requests would block the event loop of asyncio. I've found aiohttp but it couldn't provide the service of http request using ...
- 9,310
195
votes
2
answers
102k
views
What is the difference between 'content' and 'text'
I am using the terrific Python Requests library. I notice that the fine documentation has many examples of how to do something without explaining the why. For instance, both r.text and r.content are ...
- 30.2k
186
votes
3
answers
417k
views
How to send cookies in a post request with the Python Requests library?
I'm trying to use the Requests library to send cookies with a post request, but I'm not sure how to actually set up the cookies based on its documentation. The script is for use on Wikipedia, and the ...
- 14.7k
185
votes
9
answers
353k
views
How to use Python requests to fake a browser visit a.k.a and generate User Agent? [duplicate]
I want to get the content from this website.
If I use a browser like Firefox or Chrome I could get the real website page I want, but if I use the Python requests package (or wget command) to get it, ...
- 2,256
181
votes
2
answers
333k
views
Adding headers to requests module
Earlier I used httplib module to add a header in the request. Now I am trying the same thing with the requests module.
This is the python request module I am using:
http://pypi.python.org/pypi/...
- 14.9k
177
votes
11
answers
293k
views
How to get Python requests to trust a self signed SSL certificate?
import requests
data = {'foo':'bar'}
url = 'https://foo.com/bar'
r = requests.post(url, data=data)
If the URL uses a self signed certificate, this fails with
requests.exceptions.SSLError: [Errno 1] ...
- 16.8k
176
votes
1
answer
82k
views
zsh: no matches found: requests[security]
I am trying to run a python urllib2 script and getting this error:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and ...
- 1,771
175
votes
8
answers
436k
views
Python Requests and persistent sessions
I am using the requests module.
I have figured out how to submit data to a login form on a website and retrieve the session key, but I can't see an obvious way to use this session key in subsequent ...
- 3,418
163
votes
6
answers
653k
views
How do I read a response from Python Requests?
I have two Python scripts. One uses the Urllib2 library and one uses the Requests library.
I have found Requests easier to implement, but I can't find an equivalent for urlib2's read() function. For ...
- 2,277
159
votes
3
answers
340k
views
How to extract HTTP response body from a Python requests call?
I'm using the Python requests library. I'm trying to figure out how to extract the actual HTML body from a response. The code looks a bit like this:
r = requests.get(...)
print r.content
This should ...
- 5,294
158
votes
8
answers
188k
views
Log all requests from the python-requests module
I am using python Requests. I need to debug some OAuth activity, and for that I would like it to log all requests being performed. I could get this information with ngrep, but unfortunately it is not ...
- 41.5k
154
votes
6
answers
373k
views
How to "log in" to a website using Python's Requests module?
I am trying to post a request to log in to a website using the Requests module in Python but its not really working. I'm new to this...so I can't figure out if I should make my Username and Password ...
- 2,515
154
votes
9
answers
382k
views
Python requests library how to pass Authorization header with single token
I have a request URI and a token. If I use:
curl -s "<MY_URI>" -H "Authorization: TOK:<MY_TOKEN>"
etc., I get a 200 and view the corresponding JSON data.
So, I installed requests and ...
user1552586
144
votes
2
answers
202k
views
Python Requests package: Handling xml response
I like very much the requests package and its comfortable way to handle JSON responses.
Unfortunately, I did not understand if I can also process XML responses. Has anybody experience how to handle ...
- 9,503
143
votes
3
answers
222k
views
How to fix "403 Forbidden" errors when calling APIs using Python requests?
I needed to parse a site, but I got a 403 Forbidden error.
Here is the code:
url = 'http://worldagnetwork.com/'
result = requests.get(url)
print(result.content.decode())
The output is:
<html>
&...
- 1,769
143
votes
3
answers
53k
views
Difference between data and json parameters in Python Requests package
What is the difference between the data and json parameters in the Python Requests package?
It is unclear from the documentation.
Does this code:
import requests
import json
d = {'a': 1}
response = ...
- 6,023
141
votes
8
answers
300k
views
Python Requests library redirect new url
I've been looking through the Python Requests documentation but I cannot see any functionality for what I am trying to achieve.
In my script I am setting allow_redirects=True.
I would like to know ...
- 2,097
139
votes
26
answers
300k
views
Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."
I'm using requests in Python3 on Windows via Pycharm, and I'm receiving a "SSL module is not available" error.
I've spent hours trying to figure out what could be causing this. I've ...
- 1,633
137
votes
6
answers
166k
views
Why doesn't requests.get() return? What is the default timeout that requests.get() uses?
In my script, requests.get never returns:
import requests
print ("requesting..")
# This call never returns!
r = requests.get(
"http://www.some-site.example",
proxies = {'...
- 354k
135
votes
6
answers
253k
views
Download and save PDF file with Python requests module
I am trying to download a PDF file from a website and save it to disk. My attempts either fail with encoding errors or result in blank PDFs.
In [1]: import requests
In [2]: url = 'http://www.hrecos....
- 1,740
131
votes
6
answers
279k
views
How do I use basic HTTP authentication with the Python Requests library?
I'm trying to use basic HTTP authentication in Python. I am using the Requests library:
auth = requests.post('http://' + hostname, auth=HTTPBasicAuth(user, password))
request = requests.get('http://' +...
- 1,387
124
votes
4
answers
348k
views
How can I use cookies in Python Requests?
I am trying to log in to a page and access another link in the page.
I get a "405 Not Allowed" error from this attempt:
payload={'username'=<username>,'password'=<password>}
with ...
- 1,389
118
votes
9
answers
235k
views
How to make python Requests work via SOCKS proxy
I'm using the great Requests library in my Python script:
import requests
r = requests.get("http://example.com")
print(r.text)
I would like to use a SOCKS proxy, how can I do that? Requests ...
- 6,038
118
votes
5
answers
242k
views
Python Requests - How to use system ca-certificates (debian/ubuntu)?
I've installed a self-signed root ca cert into debian's /usr/share/ca-certificates/local and installed them with sudo dpkg-reconfigure ca-certificates. At this point true | gnutls-cli mysite.local is ...
- 16.8k