python how to decode http response

Question

i am using the below code to login and retrieve data from an api endpoint but seems the response see encoded and i am not able to read the content. i am using request requestes-0.0.1

import requests
import json
import os


http_proxy  = "http://192.168.10.20:8888"
https_proxy = "https://192.168.10.20:8888"

proxyDict = {
              "http"  : http_proxy,
              "https" : https_proxy,
            }
session = requests.Session()

payloadopt = 'user_id=tom&password=xxxxx'
s = session.post('https://login.milock.com/api/login',data=payloadopt, proxies=proxyDict, verify=False, headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0', 'Accept':'application/json, text/plain, */*', 'Accept-Language':'en-US,en;q=0.5', 'Accept-Encoding':'gzip, deflate, br', 'Content-Type':'application/x-www-form-urlencoded'}, stream=True)

response when i print the same on console

??7?E`??YD??k???q??f,??G?(U?Uv?4???w&??!Ψ????E5Q?_??{?F??<.????4??> p{?k9???9??

Can someone tell me how can decode and read the data from the response

Answer 1

The response is encoded with brotli compression. This compression method isn't supported by the standard library. You can install a third party package from pypi to decompress it - a number of packages are available.

For example

$ pip install brotli

>>> import brotli
>>> decompressed = brotli.decompress(response.content)
>>> dict_ = json.loads(decompressed)

If you'd prefer to avoid installing a third party module, remove 'br' from the 'accept-encoding' header in the request:

'Accept-Encoding':'gzip, deflate, br' -> 'Accept-Encoding':'gzip, deflate'