Cannot download images when using a proxy

Get Help
1

Hi all,
I’m successfully using Python to scrape websites and gather data via APIs and HTML scraping with mobile proxies. This works well, but image downloads consistently fail with error 407 (Proxy Authentication Required), even though the same proxy details work fine for data retrieval.

I’ve tried several async libraries, but none resolve the issue. Without proxy details, the images download correctly using my home IP, so I don’t believe it’s a header issue since the same headers are used.

I tested with 7 images from 3 locations (including my own server), and the result is always 407. While a non-async version works, it’s too slow for my needs.

Does anyone have ideas or working sample code for this scenario?

1 Like
2

Hey @Mexius – this sounds like a problem with your code if non-async is working. You’re likely not passing the authentication somehow.

Can you share your code snippet with us?

3

Hi Abed, thanks for responding. here is some sample code;

import aiohttp
import asyncio

async def fetch_image_aiohttp(session, url, filepath):
    proxy = "http://HOST:PORT"  
    proxy_auth = aiohttp.BasicAuth("USERNAME", "PASSWORD")  
    headers = {"Accept": "image/*"}

    try:
        async with session.get(url, proxy=proxy, proxy_auth=proxy_auth, headers=headers) as response:
            if response.status == 200:
                with open(filepath, 'wb') as f:
                    f.write(await response.read())
                print(f"Downloaded {url} to {filepath}")
            else:
                print(f"Failed to download {url}: HTTP {response.status}")
    except Exception as e:
        print(f"Error downloading {url}: {e}")

async def main():
    image_urls = [
        'https://pz-testing.b-cdn.net/300000.jpg',
        'https://media2.newlookassets.com/i/newlook/826657627/womens/clothing/coats-jackets/dark-brown-quilted-collarless-jacket.jpg?strip=true&qlt=80&w=720',
        'https://scontent.flba1-1.fna.fbcdn.net/v/t39.30808-6/470231523_1051842476988474_2806052555370673998_n.jpg?stp=dst-jpg_s600x600_tt6&_nc_cat=107&ccb=1-7&_nc_sid=127cfc&_nc_ohc=3bwazSyJG2sQ7kNvgE-dbDX&_nc_zt=23&_nc_ht=scontent.flba1-1.fna&_nc_gid=AKv8DWkEfc5cdE_Y2IXYoux&oh=00_AYDIVZxwvA-bl_g9gdqmaoBzDB0d_23WSl8wV2FDHZMXDw&oe=6772FD34',
                ]

    async with aiohttp.ClientSession() as session:
        tasks = []
        for i, url in enumerate(image_urls):
            filepath = f"image_{i}.jpg"
            tasks.append(fetch_image_aiohttp(session, url, filepath))

        await asyncio.gather(*tasks)

if __name__ == "__main__":
    asyncio.run(main())
4

oh just in case you were wondering, I obviously had real Credentials in for Host, port, UN, and pw.
thanks for any assistance.

5

Thank you. The team tested with one of our proxies and it worked fine. Do your credentials contain any special characters like a single quote?

Also, are these Proxidize proxies or self-built proxies?

1 Like
6

I used the default that was set the user name has a “hyphen-minus” (-). in it no characters. I just changed the original user name started with “proxidize-” removed the “-” and gave it a new name with no special characters and it work, thanks, that was driving me crazy last few days !

1 Like