custom headers

[unknownmenz]

hello,

I would like to know how to use custom headers. I do not know how to get it working.. (I only know how to setup a user agent so it will force to the mobile version of the website).

The reason i ask this is because if i get a certain IP assigned from the proxy, it suddenly changes in it's automated process. Which ends up banishment. As the website/account safety gets triggered when i use more as 1 IP for creating/logging in or any other activity on the website.

http://docs.proxymesh.com/article/7-request-response-headers#https

[unknownmenz]

Any help deeply appreciated
at the start of the script i get a ip assigned by connecting to the main proxy.

but for some reason the ip changes.. (i do not set any proxy in the process, only at the start).

[root]

Сhange of IP address depends on the proxy provider, as the headers are responsible for it?

[unknownmenz]

@root The headers on the link, are recommended by the proxy provider. I do not use them though. As i do not know how to implement it yet in the software.

They rotate 2 times a day at fixed times(the activity is far from those times).

[root]

@unknownmenz 0_1518831447154_test.xml So?

[daemons]

The custom headers you are talking about can help a bit, but I think mostly to an extent of a mere suggestion to the proxy service. Because they toss IPs in and out the IP pool according to their own logic, and your custom headers won't change a thing. To be honest, this type of a proxy service is not the best choice for what you want to get. There are proxy services like 3 minute or 15 minute rotating proxies. Pricing is comparable. I suspect, there are services where you can instruct the server to make a rotation when you need it by means of a custom header, just as in your case. But the prices could be "custom" as well.

[unknownmenz]

@root said in custom headers:

@unknownmenz 0_1518831447154_test.xml So?

where do i put this code?

[unknownmenz]

@daemons

I need to connect to one host proxy and it will assign me 1 random ip address from the pool, which is available for me (depends on country/size of package).

Let's say i have a package of 10 IP's.
if i assign to the host proxy, i will get one of the 10 IP's.
It won't change except on 17:00 evening and 05:00 morning. Unless i try to connect/request to the host proxy again.

[daemons]

@unknownmenz Just place anywhere before the Load action.

[unknownmenz]

@daemons i did that, setting the proxy is the first thing which is done before any other acitivty will be done.

[daemons]

@unknownmenz Good. Browser\Proxy, then Network\Set Header, then Browser\Load should all get the work done. Use Fiddler to double check.

[unknownmenz]

is there anything i need to put into "set header"? because i use it know so it forces X website to mobile device website

edit: so it visits a website like a android device (user agent)

[daemons]

@unknownmenz Just set X-ProxyMesh-IP equal to the IP from the preceding response header's X-ProxyMesh-IP. That should be it. According to your ProxyMesh docs. All X-headers are just a way to separate them from the standard. There's nothing to do with desktop or mobile. Servers only can determine that by the User-Agent string. I say it again, use Fiddler to double check everything, including the User-Agent headers. Then tweak if needed.

[unknownmenz]

@daemons Thanks a lot, i will try to get it solved. I will let you know.

[unknownmenz]

@daemons It seems not to work for me, it looks like everytime i use the load function another ip gets assigned from the pool.. with or without the headers i try to use.

And fiddler i try to work it out as i am new to a program like that.

[daemons]

@unknownmenz Probably it's time to contact their support. But anyway they'd need some logs. Fiddler is a way to go then. It's widespread, learn it.

BTW there are many other ways for servers to detect unwanted clients, not only headers, but first you need to get rid of your IP problem.

[unknownmenz]

@daemons said in custom headers:

@unknownmenz Probably it's time to contact their support. But anyway they'd need some logs. Fiddler is a way to go then. It's widespread, learn it.

BTW there are many other ways for servers to detect unwanted clients, not only headers, but first you need to get rid of your IP problem.

This is their response.

"
Another method for controlling the IP addresses is to use random string in the Proxy-Authorization header. The way it works is instead of passing in an encoded username:password in the authentication header, you pass in username:string:password, where string is a random string that you generate. As long as you keep using the same string, the proxy server will choose the same outgoing IP, as long as that IP is available. If the IP is no longer available, then a new IP will be chosen by the proxy server, and will be used for subsequent requests with the same string. Most client libraries support the Basic access authentication method, which means the only change required in your code, is to join the string to your username when providing the header value. However, for HTTPS requests, only some libraries support sending the Proxy-Authorization header with the initial CONNECT, such as Python requests. Other client libraries may require custom code to make the Proxy-Authorization header work for HTTPS requests.
"
But i do not understand it.. too technical for me.. could you explain it to me how to implement it? or explain in baby talk ?

(im so sorry for the bothering).

[daemons]

@unknownmenz Long story short, I've tested their approach on a dummy HTTPS site (Google) and found out that BAS's browser (it's a custom build of Chrome) does not support Proxy-Authorization headers in HTTPS CONNECT, but HTTP-client (it's curl-based) does.

You can find the two test cases attached 0_1518990005633_tests01.xml and 1_1518990005633_tests02.xml. The first one lacks any auth data in Fiddler logs at all, other one shows what we need (last row):

CONNECT www.google.com:443 HTTP/1.1
Host: www.google.com:443
Connection: Keep-Alive
Proxy-Authorization: Basic dXNlcjoyczUwNjc1cTpwYXNz

So, in fact, it all depends on your goals and resources. If you can build your app around GET/POST requests then the above approach would work in BAS. But if the app depends on Javascript, user interaction, etc. then no.

But I think, the first approach with X-headers is not completely compromised yet. So did you study Fiddler logs?

[unknownmenz]

@daemons said in custom headers:

@unknownmenz Long story short, I've tested their approach on a dummy HTTPS site (Google) and found out that BAS's browser (it's a custom build of Chrome) does not support Proxy-Authorization headers in HTTPS CONNECT, but HTTP-client (it's curl-based) does.

You can find the two test cases attached 0_1518990005633_tests01.xml and 1_1518990005633_tests02.xml. The first one lacks any auth data in Fiddler logs at all, other one shows what we need (last row):

CONNECT www.google.com:443 HTTP/1.1
Host: www.google.com:443
Connection: Keep-Alive
Proxy-Authorization: Basic dXNlcjoyczUwNjc1cTpwYXNz

So, in fact, it all depends on your goals and resources. If you can build your app around GET/POST requests then the above approach would work in BAS. But if the app depends on Javascript, user interaction, etc. then no.

But I think, the first approach with X-headers is not completely compromised yet. So did you study Fiddler logs?

I just need to make sure i keep using the same Proxy IP while doing its process on a https website.

I wasn't be able to learn fiddler yet(even though the the program looks cool, the program makes me feel like a hacker LOL).

[daemons]

@unknownmenz That's what I'm talking about: the process. It all depends on it. If it allows sticking to GET/POST - then OK, go with BAS. If you need complicated user interaction stuff - then sorry no. But that's for the second approach. First you need to vet the first one. I'd be happy to help you more, but I think, it's not possible without further details on you task.