Skip to content

Conversation

@codinglearner121
Copy link

What changed?

  1. Extended HTTP headers to fully simulate a browser request (added Accept, Accept-Language, etc.), solving 403 Forbidden caused by incomplete headers.
  2. Added retry logic for HTTPError: If the server rejects Accept-Encoding, automatically retry without that header.
  3. Ensured the target directory exists before saving files, avoiding path-related errors.

Why?

I encountered 403 Forbidden errors when downloading files—some servers block requests with incomplete headers. These changes make the download function more robust and compatible, fixing the access issue while improving error handling.

Tested on

  • URLs that previously returned 403 (now download successfully).
  • Paths with non-existent directories (auto-created and saved correctly).

@codinglearner121
Copy link
Author

The 403s were happening because the old request lacked browser-grade headers (Accept, Accept-Language, etc.). Servers often block “script” traffic. I added Chrome-like headers and a fallback that strips gzip when the server chokes on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant