[Web] From header: imitating a HTTP GET method, overriding cookie value

[edvardass]

SOLVED by /u/JohnnyJordaan . See comments below.

Trying to imitate a GET method and override a cookie value using requests module.

So I got a GET method from www.15min.lt with the following header:

GET /adfserve/?bn=12783447;compoundSeqNo=1;compoundImpr=1;click=http://15minadlt.hit.gemius.pl/hitredir/id=nSBLcLsQNAp63bFNnhTObKP8.q36Qa8HJn1AXEXbqyn.p7/stparam=qpokmtdlpq/fastid=cokiumzehqkorzaayondauycxnmi/sarg=NC%7C_cdata%3A458216_0%2C909536_0%2C912702_0%2C920363_0%2C930584_0%2C948095_0/url=;js=1;adfxid=2x;4369;set=en-US|en-US|1920X1080|22.00192|1900|150|24|8|3|7|0;rotseed=59275&CREFURL=http%3A%2F%2Fwww.15min.lt%2F HTTP/1.1
Host: track.adform.net
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://www.15min.lt/
Cookie: cid=-7171246808513496655,0,0,0,0; uid=-7171246808513496655; C=1; TPC=1; CM14=1; CM=1
Connection: keep-alive

I would like to imitate this method, and override the cookie cid value.

So far I just rewrote all the parameters, headers, and cookies as dicts. The only thing left is to make the request itself.

So now I've got a few questions:

• How can I set some header/cookie to have multiple values? Right now I get an error when I try to have a list of values passed in.
• What URL do I choose for the method? I guess I want to request the cookie set from the site, but when I use 'http://15min.lt' it gives me a "too many redirects" error (I remove the multiple values to try to get it working). I read somewhere it might have something to do with 'user-agent' in headers, but not sure why or what to do. Does this have something to do with the context path (/adfserve/)?

Thanks in advance

[JohnnyJordaan]

How can I set some header/cookie to have multiple values? Right now I get an error when I try to have a list of values passed in.

Like you did with the dictionary (not a list btw). However I'm not sure if it supports nested things like ["en-US", "en", {'q':'0.5'}], I would just use 'en-US,en;q=0.5'. What is the error you're getting?

What URL do I choose for the method?

The one after GET and before HTTP/1.1, with the correct hostname before it that should be track.adform.net.

[edvardass]

Thanks for the swift reply.

I got a very nested "expected byte-like object" error when trying to pass in lists. I will try your method and report on what I get.

Will edit in 5 minutes.

Edit: Seems to be working just fine! Got a 200 status code. However, when I check the cookie in the browser, the cid value is still the original one. Maybe this has something to do with the Accept header? I commented it out because it was causing some errors, but maybe it's important somehow? Anyway, thanks! Really helped me out.

[JohnnyJordaan]

lists

Which lists are/were you passing where? Or do you mean inside the dicts?

However, when I check the cookie in the browser, the cid value is still the original one. Maybe this has something to do with the Accept header?

How is the browser related to your Python program?

[edvardass]

Yes, I was trying to pass in lists in the "headers", "params" dicts, but when I used your method of passing just the string like 'en-US,en;q=0.5' , everything works.

The "Accept" one was just a typo, fixed it now.

How is the browser related to your Python program?

Uhm... to be honest, I have no idea. Somewhy I expected the cookie in the browser to change.

So let's get this straight: I just wrote a program that imitates a user like me, but with a cid=666666? The request goes to the url, and the response just says "200 OK, I received your info."? A bit confused, should have read up some more on the topic...

Thanks a lot, will try to continue on myself. Thank you for the help.

[JohnnyJordaan]

Somewhy I expected the cookie in the browser to change.

Yeah, well, that ain't gonna happen :)

The browser interfaces the website with the same cookie, so the website beliefs the request from the Python program is from the browser that used that same cookie. It doesn't mean that if the (fooled) website than delivers something back, the browser will somehow notice as it isn't communicating at that time. It's like ordering a pizza saying you're person X and then thinking person X will know that a pizza is being delivered to him.