Cookies expiring when changing IPs?

[ne1baf]

DO NOT DELETE THIS! Please take the time to fill this out properly. I am not able to help you if I do not know what you are executing and what error messages you are getting. If you are having problems with a specific video make sure to include the video id.

I'm using AWS Lambda to fetch transcripts, I recall this working correctly (not expiring after an IP change) with cookies a month or so ago. But now cookies seem to expire on IP change?
Note that if I hit refresh youtube, get a new cookie file and update the cookie file on my lambda and re-run the code, it will work again.

To Reproduce

• get cookies with Cookie-Editor extension
• run lambda code to fetch transcript with the cookies file (this should work)
• wait 10 to 20 mins so that the instance disconnects
• try running the same code again to fetch transcripts (this shouldn't work anymore)

What code / cli command are you executing?

For example: I am running

YouTubeTranscriptApi.get_transcript("yTocZOLjzC8", languages=['fr', 'en'], cookies='cookie.txt')

Which Python version are you using?

Python 3.11 on AWS Lambda

Which version of youtube-transcript-api are you using?

youtube-transcript-api 0.6.2

Expected behavior

I expected to receive the transcript

Actual behaviour

For example: Instead I received the following error message:

Could not retrieve a transcript for the video https://www.youtube.com/watch?v=yTocZOLjzC8! This is most likely caused by:

Subtitles are disabled for this video

If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem!
2024-11-13 09:22:42,541 - DEBUG - Response content analysis: playerResponse: True, captions: True, timedtext: False
Result: {'statusCode': 404, 'body': 'Failed to fetch transcript'}

[asonawalla]

FWIW, I'm seeing the same behavior in google cloud run (GCP's lambda equivalent) this week. I don't use the cookies feature at all and it was working consistently up to last week.

[jdepoix]

Hi, this is very unlikely to be related to your cookies expiring, but more to YouTube blocking your lambdas IP address (look at #303 to find out more about this). Do you observe the same behaviour if not using cookie at all?

@asonawalla in your case your most definitely experience the issue described in #303.

[ne1baf]

I don't think it's about the lambda IP changing as I mentioned in my original message.

hi @jdepoix, yes I had a look at #303 but in my case trying to fetch transcripts from my lambda doesn't work if I'm not using cookies. However if I add cookies, I'll be able to make successful requests but after 10/20 mins, that cookie is not useable anymore and requests are blocked again. If I get a fresh cookie (even if it's from the same YT account as before), I'll be able to once again make successful requests for 10/20 mins.

I also tried this with a proxy that blocked by YT. Requests did not go through without using the cookie, however, after adding the cookie, requests would go through for 10/20 mins again before being blocked again.

Using another proxy without cookie fixed this for me but I used to be able to make requests with cookies without them "expiring" after 20 minutes and this behavior seems to have changed.

[asonawalla]

@jdepoix yeah, I figured. I'm running it on a non-cloud server without any issues now. Thanks for your work on this!

[peperojel]

this is not clear enough to reproduce

get cookies with Cookie-Editor extension
run lambda code to fetch transcript with the cookies file (this should work)

you need to specify how you are providing Cookies to Lambda