#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1

    Recommend A Webhost To Run Open Public Proxy-What To Look-out For When Running One ?


    Folks,

    I am looking for a free or a paid webhost that will allow me to run my own web proxy for the public to use. I am not looking to use a third party service. No! I want to provide the proxy service to the public (like to you) like anonymouse.org does.

    Q1. I'm going to use Mini Proxy php script to run the service. Is this Mini Proxy safe to use ?
    https://github.com/josh****/miniProx.../miniProxy.php

    Q2. And, I've searched google twice before few wks ago on 2 different days but no luck. The webhosts I usually find, will not allow me to run my own public proxy service. Some will not allow me to run it on a shared server and expect me to hire a dedicated server and charge me an arm & leg every month. That's the big problem.
    I've used many keywords in the google search to find the right host. But, no luck.
    Can you think of a few key-phrases that will yield positive results on the google search ?
    The host must allow me to use port 3306 so that my .exe bots can dump data to my website's mysql database.

    Q3. You know I have been googling to learn the risks of running your own open public proxy but no luck in finding any link that spells-out all the risks involved. Do you know of any good link ?

    Q4. Is it necessary to host your web proxy on httpS (SSL) ?

    Q5. Is it necessary to buy a httpS (SSL) certificate for my domain or website ?

    Q6. What resources must I get from my webhost as a minimum ?
    I was told to get the following:

    * An Apache server with at least PHP 5 installed, along with cURL support.
    * Write access to public_html.
    * The ability to set up a proxy.

    Is there anything else you'd like to add in that list that I must get as a minimum from my webhost to run my own open public web proxy ?


    Thank You
    Last edited by UniqueIdeaMan; October 22nd, 2017 at 08:29 PM.
  2. #2
  3. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Look at a low-cost VPS. That will let you start cheaply, and you'll have as much control over it as you need. Then as you see it become too slow you can upgrade to a plan with higher resources

    Just remember that a public proxy like that uses a whole lot of resources and processing power, so don't be amazed if a very small handful of users cripples a relatively decent VPS. That's why hosting companies don't allow you to run things like that on a shared platform. It uses way to many resources and would leave every other site on the server lagged or just plain broken.

    Comments on this post

    • UniqueIdeaMan agrees
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1
    Originally Posted by Catacaustic
    Look at a low-cost VPS. That will let you start cheaply, and you'll have as much control over it as you need. Then as you see it become too slow you can upgrade to a plan with higher resources

    Just remember that a public proxy like that uses a whole lot of resources and processing power, so don't be amazed if a very small handful of users cripples a relatively decent VPS. That's why hosting companies don't allow you to run things like that on a shared platform. It uses way to many resources and would leave every other site on the server lagged or just plain broken.
    Thanks man! What you say is true. I figured all this myself.
    Anyway, how-about answering my 5 questions as much as you can ? Been googling for answers but no luck.
    Frankly, I trust your advices, judgments, conclusions, criticisms and opinions. So, whatever answer you give as a reply has more weight than what I'm likely to find googling as they could be written by a bunch of no good amateurs. And, I KNOW you are not.

    Cheers!
  6. #4
  7. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Your questions? OK, lets try it...

    Q1. I'm going to use Mini Proxy php script to run the service. Is this Mini Proxy safe to use ?

    I can't tell, and I'm not going to spend the time going thorugh the whole thing line by line to check. Just remember that every bit of woftware ever written will have *some* possible vunerability in it. The only thing that's different is how long it takes someone with bad intentions to find and exploit that vunerability.

    Q2. And, I've searched google twice before few wks ago on 2 different days but no luck. The webhosts I usually find, will not allow me to run my own public proxy service. Some will not allow me to run it on a shared server and expect me to hire a dedicated server and charge me an arm & leg every month. That's the big problem.

    You've answered your own question. Shared servers don't allow it, so you have to look for a different type of hosting account.

    Q3. You know I have been googling to learn the risks of running your own open public proxy but no luck in finding any link that spells-out all the risks involved. Do you know of any good link ?

    If you can't see the risks for yourself, you should not be operating a public proxy. You will have problems, you will get attacked, and you will most likely get hacked, and probably badly.

    Q4. Is it necessary to host your web proxy on httpS (SSL) ?

    No. But it's usefull for people to take your "service" seriously.

    Q5. Is it necessary to buy a httpS (SSL) certificate for my domain or website ?

    If your host offers Lets Encrypt certificates then use those. If not you will need to purchase one. A self-signed certificate is worse then no certificate at all.

    Q6. What resources must I get from my webhost as a minimum ?

    No idea, and to be honest no one can tell you. To find that out you need to run the service and see what fails. When something fails, upgrade to give that part more resources.
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1
    Originally Posted by Catacaustic
    Your questions? OK, lets try it...

    Q1. I'm going to use Mini Proxy php script to run the service. Is this Mini Proxy safe to use ?

    I can't tell, and I'm not going to spend the time going thorugh the whole thing line by line to check. Just remember that every bit of woftware ever written will have *some* possible vunerability in it. The only thing that's different is how long it takes someone with bad intentions to find and exploit that vunerability.

    Q2. And, I've searched google twice before few wks ago on 2 different days but no luck. The webhosts I usually find, will not allow me to run my own public proxy service. Some will not allow me to run it on a shared server and expect me to hire a dedicated server and charge me an arm & leg every month. That's the big problem.

    You've answered your own question. Shared servers don't allow it, so you have to look for a different type of hosting account.

    Q3. You know I have been googling to learn the risks of running your own open public proxy but no luck in finding any link that spells-out all the risks involved. Do you know of any good link ?

    If you can't see the risks for yourself, you should not be operating a public proxy. You will have problems, you will get attacked, and you will most likely get hacked, and probably badly.

    Q4. Is it necessary to host your web proxy on httpS (SSL) ?

    No. But it's usefull for people to take your "service" seriously.

    Q5. Is it necessary to buy a httpS (SSL) certificate for my domain or website ?

    If your host offers Lets Encrypt certificates then use those. If not you will need to purchase one. A self-signed certificate is worse then no certificate at all.

    Q6. What resources must I get from my webhost as a minimum ?

    No idea, and to be honest no one can tell you. To find that out you need to run the service and see what fails. When something fails, upgrade to give that part more resources.

    Thanks Catacaustic, your reply was encouraging.

    As for the risks involved with running a web proxy server, I can think of the following at the top of my head. I'd appreciate it if you can add your own risk possibilities and grow the list.

    * Too many users simultaneously using the service - (slowing down your proxy server)
    * Illegal Sites Browsing - (getting your proxy server IP in trouble with the law)
    * Viewing Streaming videos - (slowing down your proxy server and draining the bandwidth and causing trouble for other users such as timeouts)
    * Bulk Downloads (videos, music files) - (slowing down your proxy server and draining the bandwidth and causing trouble for other users such as timeouts)
    * Bulk Uploads (videos, music files) - (uploading virus & distributing it, slowing down your proxy server and draining the bandwidth and causing trouble for other users such as timeouts)
    * DOS Attack - (getting your proxy server IP in trouble with the laws)
    * Pirate Software Downloads - (getting your proxy server IP in trouble)

    But, I don't think I would have any of the following risks by running a web proxy unless I run a Socks Proxy, right ?

    * Spamming via your SMTP (unless, I run a webmail)
    * Torrenting (Mas Seeding)
    * Spreading Virus & Malware (unless, I allow uploading to my site/server)

    What do you say ?
    Anyway, do you know what kind of hacking risks there are by running a web proxy (if there are any risks, that is) ?
    Last edited by UniqueIdeaMan; November 6th, 2017 at 03:44 PM.
  10. #6
  11. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Originally Posted by UniqueIdeaMan
    But, I don't think I would have any of the following risks by running a web proxy unless I run a Socks Proxy, right ?

    * Spamming via your SMTP (unless, I run a webmail)
    * Torrenting (Mas Seeding)
    * Spreading Virus & Malware (unless, I allow uploading to my site/server)

    What do you say ?
    You'll always have a risk of getting your SMTP hacked/abused unless you disable it or remove it. if you do that you won't be able to send any emails at all from the server.

    Torrenting will most likely not happen unless you open up the right ports for it.

    Spreading virus and malware could happen at any time. Not only from your server, but if any site that you're proxy hits is spreading that, then you'll be spreading that too.

    You need to either learn a whole lot more about running a server, or pay someone to do it for you. If you don't you will get pwned, and pretty quickly.

    Originally Posted by UniqueIdeaMan
    , do you know what kind of hacking risks there are by running a web proxy (if there are any risks, that is) ?
    There's many, many risks. We've seen your code, and as much as I'm not trying to be mean here, just realistic, you don't even really understand the basics of how it works, let alone have any idea of how to deal with higher-level security issues. That's a recipe for disaster. At best your proxy will get hacked and used for purposes that you don't want it to. At worst it will get hacked and made to do the wrong thing and you'll be responsible for it. Depending on exctly how badly it's hacked will depend on what ramifications there are. No one here could possibly tell you that, so you'll have to decide if the risk is small enough for you to allow it.
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1
    Originally Posted by Catacaustic
    You'll always have a risk of getting your SMTP hacked/abused unless you disable it or remove it. if you do that you won't be able to send any emails at all from the server.

    Torrenting will most likely not happen unless you open up the right ports for it.

    Spreading virus and malware could happen at any time. Not only from your server, but if any site that you're proxy hits is spreading that, then you'll be spreading that too.

    You need to either learn a whole lot more about running a server, or pay someone to do it for you. If you don't you will get pwned, and pretty quickly.



    There's many, many risks. We've seen your code, and as much as I'm not trying to be mean here, just realistic, you don't even really understand the basics of how it works, let alone have any idea of how to deal with higher-level security issues. That's a recipe for disaster. At best your proxy will get hacked and used for purposes that you don't want it to. At worst it will get hacked and made to do the wrong thing and you'll be responsible for it. Depending on exctly how badly it's hacked will depend on what ramifications there are. No one here could possibly tell you that, so you'll have to decide if the risk is small enough for you to allow it.
    If I don't run a SOCKS proxy but an http proxy then how will anyone spam through my proxy ? I won't run my own webmail and I'll blacklist hotmail, yahoomail, etc. Nor, my own SMTP.
    As for DDOS ATTACK, I can put a limit to how many times the same page is fetched by the same user within the same minute. That should prevent any DDOS ATTACKs.

    What do you say to my abuse foiling methods ? I'm not arguing with, I'm learning from you if my methods are reasonable or not and of not then need an explanation to why they are not.
  14. #8
  15. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Originally Posted by UniqueIdeaMan
    If I don't run a SOCKS proxy but an http proxy then how will anyone spam through my proxy ? I won't run my own webmail and I'll blacklist hotmail, yahoomail, etc.
    HTTP only doesn't mean no SPAM. It's just not directly sent from your service. There's many ways that hackers can crack your code to send emails out from it.

    Originally Posted by UniqueIdeaMan
    As for DDOS ATTACK, I can put a limit to how many times the same page is fetched by the same user within the same minute. That should prevent any DDOS ATTACKs.

    What do you say to my abuse foiling methods ? I'm not arguing with, I'm learning from you if my methods are reasonable or not and of not then need an explanation to why they are not.
    DDOS won't be mitigated by a limit like that. The connection attempts will continue, and your server will still deal with them, even if it's not processing everything completely. Setting just a per-page limit is pretty much useless when you get 10,000,000 connections requesting 1,000,000 different URL's each.

    Comments on this post

    • UniqueIdeaMan agrees
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1
    Catacaustic,

    I just read your previous reply. Was offline for half a wk.

    How come public web proxies don't get hacked then or get used for illegal purposes ?
    For example anonymouse.org been running for over a decade.
    Last edited by UniqueIdeaMan; November 6th, 2017 at 03:37 PM.
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1
    Catacaustic,

    I just read your previous reply. Was offline for half a wk.

    I want to know from you, since I want to run a public proxy then would I still become a target like you say even after adding the following measures ? If so, then how to prevent all this from happening ? The following are the steps I am taking. Do you mind checking if they are safe & sound or not ?

    1. Not run an SMTP. This is to prevent anyone using my mail server†to spam;

    2. Publicise the user's IP along with the links he visits†to prevent anyone from daring to view illegal sites or upload anything malicious or download anything illegal;

    3. Force the user to open an account under a username that matches his domain name and log his username/domain along with the links he visits. Example:

    Time|IP|Username|KW Searched|Visited Page

    02:59pm|143.133.135.138|devshed.com|php 7 books|php-book.com

    During registration, I would get the php script to prompt the user to submit an email address under his domain name. The email would contain his account activation link. That is how I would make sure that it is Tom Boy who is submitting tom@tomboy.com and not any **** & Harry.†

    4. Added a Banned Words Filter that replaces banned words on a proxied page. Will change this to add instead a feature that prevents loading a page that contains banned words.

    5. Add a php function†in the proxy script†to block file downloads. That should prevent anyone downloading related to illegal stuffs.

    Example, the proxy would replace .mp3, mp4 extensions from links. Change:

    https://www.devshed.com/download/php.mp4

    to:

    https://www.devshed.com/download/php.***

    That way, any link containing a downloadable extension would not be fetched by the proxy.

    Do you think this tactic would work to prevent downloads ?

    6. Add php function(s)†in the proxy script†to block audio/video streaming. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.

    7. Add php function(s)†in the proxy script†to block uploads. That should prevent anyone uploading any files related to illegal stuffs.



    QUESTIONS

    Q1. So, what is that†php function that blocks downloads†? That should prevent anyone downloading malware/viruses, etc. using my proxy ?

    Q2. And, what is that†php function that blocks audio/video streaming (downloading) ? That should prevent anyone viewing video streams using my proxy ?

    Q3. And, what is that†php function that blocks uploads†? That should prevent anyone uploading malware/viruses, etc. using my proxy.

    Q4. And, what is that†php function that blocks audio/video streaming (uploading) ? That should prevent anyone uploading viruses infected video files, etc. using my proxy.

    Q5. And, what is that php function that prevents the user's browser from playing any audio/video files on a website ? Eg. Prevent playing youtube vids, vimeo vids, metacafe vids, clickbank vids,†etc. ?

    Q6. And,†what is that php function that records bandwidth usage (uploads & downloads and audio/video streaming) ? I might aswell give each account just enough data limit for them to browse text pages but not enough limit to view or listen to audio/video pages (like youtube vid pages).

    Q7. Any other features to add to prevent anyone abusing my public proxy service ? If so, which php functions should I use to add them ?
    Last edited by UniqueIdeaMan; November 6th, 2017 at 03:58 PM.
  20. #11
  21. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Originally Posted by UniqueIdeaMan
    I want to know from you, since I want to run a public proxy then would I still become a target like you say even after adding the following measures ?
    Nope. I would never use a service like that. Especially with some of your "ideas" below. I don't want everyone out there knowing what I'd doing. Adveritsing tracking is bad enough these days without making all of that publicly identifiable to anyone that wants to see it.

    Originally Posted by UniqueIdeaMan
    1. Not run an SMTP. This is to prevent anyone using my mail server†to spam;
    Will be OK, but there's other ways to hack your server to send SPAM. But mostly it's not the sending that's the problem, It's hosting or showing sites that are landing pages for the SPAM. Your system will allow that all day every day.

    Originally Posted by UniqueIdeaMan
    2. Publicise the user's IP along with the links he visits†to prevent anyone from daring to view illegal sites or upload anything malicious or download anything illegal;
    Terrible idea, and it won't stop anyone. Anyone that would care about this would a: be using a VPN, or TOR, or soemthing els, and b: wouldn't be using a "service" like this in the first place.

    Originally Posted by UniqueIdeaMan
    3. Force the user to open an account under a username that matches his domain name and log his username/domain along with the links he visits. Example:

    Time|IP|Username|KW Searched|Visited Page

    02:59pm|143.133.135.138|devshed.com|php 7 books|php-book.com

    During registration, I would get the php script to prompt the user to submit an email address under his domain name. The email would contain his account activation link. That is how I would make sure that it is Tom Boy who is submitting tom@tomboy.com and not any **** & Harry.†
    Yeah... that will verify that someone has an email account with that domain. That doesn't mean a whole lot though. Anyone could open up any URL for an anonymous webmail system, so you'd be verifying domains all day long.

    Originally Posted by UniqueIdeaMan
    4. Added a Banned Words Filter that replaces banned words on a proxied page. Will change this to add instead a feature that prevents loading a page that contains banned words.
    Useless. Look up "clbuttic" and you'll see why.

    Originally Posted by UniqueIdeaMan
    5. Add a php function†in the proxy script†to block file downloads. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.

    Example, the proxy would replace .mp3, mp4 extensions from links. Change:

    https://www.devshed.com/download/php.mp4

    to:

    https://www.devshed.com/download/php.***

    That way, any link containing a downloadable extension would not be fetched by the proxy.

    Do you think this tactic would work to prevent downloads ?
    And just how do you know that a link points to a downloadable file? The only way todo that is to filter file extensions, but that won't work with a lot of redirection scripts and tracking scripts as they don't use the file extension anyway.

    Originally Posted by UniqueIdeaMan
    6. Add php function(s)†in the proxy script†to block audio/video streaming. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.
    Again, how would you know that something is a stream? Also, there's a whole lot of aidio and video streaming that's not illegal. Are you going to block that too? (hint, you should, or you'll be paying millions for bandwidth. Think about what will happen when someone starts using your service to watch high-res YouTube videos for 3-4 hours a day)

    Originally Posted by UniqueIdeaMan
    7. Add php function(s)†in the proxy script†to block uploads, block file downloads and prevent streaming. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.
    How would blocking uploads block anything illegal?

    Originally Posted by UniqueIdeaMan
    Q1. So, what is that†php function that blocks downloads†? That should prevent anyone uploading malware/viruses, etc. using my proxy ?
    PHP doesn't block things like that. I can write something yourself that can check some things, but you'd need to do that yourself to fit in with your system.

    Originally Posted by UniqueIdeaMan
    Q2. And, what is that†php function that blocks audio/video streaming (downloading) ? That should prevent anyone uploading malware/viruses, etc. using my proxy ?
    See above...

    Originally Posted by UniqueIdeaMan
    Q3. And, what is that†php function that blocks uploads†? That should prevent anyone uploading malware/viruses, etc. using my proxy.
    Again, see above...

    Originally Posted by UniqueIdeaMan
    Q4. And, what is that†php function that blocks audio/video streaming (uploading) ? That should prevent anyone uploading malware/viruses, etc. using my proxy.
    Again, see above (get the pattern here?)...

    Originally Posted by UniqueIdeaMan
    Q5. And, what is that php function that prevents the user's browser from playing any audio/video files on a website ? Eg. Prevent playing youtube vids, vimeo vids, metacafe vids, clickbank vids,†etc. ?
    See above...

    Originally Posted by UniqueIdeaMan
    Q6. And,†what is that php function that records bandwidth usage (uploads & downloads and audio/video streaming) ? I might aswell give each account just enough data limit for them to browse text pages but not enough limit to view or listen to audio/video pages (like youtube vid pages).
    There arent any. You'd need to keep the bandwidth logs on the server and parse them to get all of that information. As a warning, that job is a whole lot harder than it sounds.

    Originally Posted by UniqueIdeaMan
    Q7. Any other features to add to prevent anyone abusing my public proxy service ? If so, which php functions should I use to add them ?
    Nope. You'll be hacked in a month anyway, or you'll be shut down for all of the bad links that you're making available, so there's not much point trying to build in anything extra.

    What it all comes down to is this:

    You have got 0 ide aon how to run a proxy, you don't know how to control it, and you don't understand the issues behind it or how anything works, so you are putting yourself in for anything from a bad site to getting shut down qickly, to getting sued for scraping other sites content, and possibly even arrested for whatever illegal stuff will end up happening through your system. Unless you can get some really great legal advice that says that it's all OK, drop this idea now because the pittiance of money that you'll make from it won't even pay for an initial consultation from the lawyer that you will need.
  22. #12
  23. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1

    Question


    Originally Posted by UniqueIdeaMan
    1. Not run an SMTP. This is to prevent anyone using my mail server to spam;
    Originally Posted by Catacaustic
    Will be OK, but there's other ways to hack your server to send SPAM. But mostly it's not the sending that's the problem, It's hosting or showing sites that are landing pages for the SPAM. Your system will allow that all day every day.
    Q1. What do you mean by "landing pages for the SPAM" ?

    Q2. I notice Mini Proxy has a feature where I can either hide the user's ip or reveal it to the websites the user visits.
    I'll just opt for the latter. That should deter criminals, hackers & spammers.


    Originally Posted by UniqueIdeaMan
    2. Publicise the user's IP along with the links he visits to prevent anyone from daring to view illegal sites or upload anything malicious or download anything illegal;
    Originally Posted by Catacaustic
    Terrible idea, and it won't stop anyone. Anyone that would care about this would a: be using a VPN, or TOR, or soemthing els, and b: wouldn't be using a "service" like this in the first place.
    That is my point. Hackers, criminals and spammers want to remain anonymous. They don't want to be identified nor publicise their internet activities. Since my "public proxy" would not be providing an "anonymous browsing" service but "browsing publicising" service instead then why should my public service or public proxy attract Hackers, criminals and spammers ?
    You DO have a question to why anyone would use my public proxy and so I'll be a little brief so you are not left in the dark nor keep thinking I'm a fool who does not know what he is talking about.
    I can't simply reveal all my ideas here in public if I want others to not copy my idea and launch the program before I can. Let's just say I have 4 groups of people all over the world (a new industry) who want to publicise what they are viewing for commercial, learning, teaching or shopping purposes. When I say "commercial", I mean they would earn money when others view their browsing history. That is their incentive to publicise what they are viewing. They'd be in control to what they publicise and what they don't. They can delete the links later that they publicised earlier on auto mode. In short, just imagine 4 groups of people all over the globe have very good reasons to publicise from time to time what they are viewing.
    Now, I need to cater to their needs. I first wanted to provide a top & bottom framed page (their homepage after they log-in to their account) where they view websites on the bottom frame while the top frame logs the pages they are viewing. That way, there would be no need for my server to fetch pages for them and there would be no risk of anyone using my server to upload and/or download anything malicious. But, html 5 has deprecated the frame & iframe. And so, I was stuck on how to track & log what others browse for their browsing to be made public like they want. At the end, I realised a web proxy has the capacity to log proxified pages. Mini Proxy (Gpl) does not log proxified pages and so I added my own. Built my own account reg, acc login, acc homepage, acc logout pages. Now only logged-in users can use the Mini Proxy (public proxy).
    Now, before I open the service to the public, I just want to make sure no one would abuse the service. I know that, if I provide any form of anonymity feature then that would attract exploiters. Therefore, best I don't promote my service as a proxy but a "publicising service" instead. That way, the good public (who are privacy conscious) and the bad public (anonymous criminals) don't find any interest in it but those 4 concerned parties only. I also need to take proper measures so copy cats don't try abusing my servers to get a competition out of business.

    After reading my previous post, you came to the conclusion that a privacy conscious person (the general public) would not be interested in my public proxy. Good.
    But, after reading all this in this post, do you still reckon the criminals would find interest in my public proxy or service ? I'm using prepared statements and so that should prevent hacking. Right ?
    It all boils down to this:
    I need to only attract those who want to publicise what they are browsing and their activities on my proxy service should not land me in trouble with any law, any website, my isp and my webhost. So, what measures do you suggest I take now on this journey to prevent any member abusing the service and getting both himself & me in trouble ?


    Originally Posted by UniqueIdeaMan
    3. Force the user to open an account under a username that matches his domain name and log his username/domain along with the links he visits. Example:

    Time|IP|Username|KW Searched|Visited Page

    02:59pm|143.133.135.138|devshed.com|php 7 books|php-book.com

    During registration, I would get the php script to prompt the user to submit an email address under his domain name. The email would contain his account activation link. That is how I would make sure that it is Tom Boy who is submitting tom@tomboy.com and not any **** & Harry.
    Originally Posted by Catacaustic
    Yeah... that will verify that someone has an email account with that domain. That doesn't mean a whole lot though. Anyone could open up any URL for an anonymous webmail system, so you'd be verifying domains all day long.
    What do you mean ? I don't understand your hint. The whole purpose of identifying the user (account holder) via his domain name is so that no user submits his free email address (hotmail, etc.) but an email belonging to his domain name so he knows his internet activities are getting logged under his domain name and if he tries abusing the service then the law will trace him via his domain registrar and prosecute him.
    The email address verification would be done by the script. Not by me manually.


    Originally Posted by UniqueIdeaMan
    4. Added a Banned Words Filter that replaces banned words on a proxied page. Will change this to add instead a feature that prevents loading a page that contains banned words.
    Originally Posted by Catacaustic
    Useless. Look up "clbuttic" and you'll see why.
    I don't need to. Look at the answers here:
    https://stackoverflow.com/questions/...in-other-words
    So, now you agree I took care of this pitfall you are fearing ?


    Originally Posted by UniqueIdeaMan
    5. Add a php function in the proxy script to block file downloads. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.

    Example, the proxy would replace .mp3, mp4 extensions from links. Change:

    https://www.devshed.com/download/php.mp4

    to:

    https://www.devshed.com/download/php.***

    That way, any link containing a downloadable extension would not be fetched by the proxy.

    Do you think this tactic would work to prevent downloads ?
    Originally Posted by Catacaustic
    And just how do you know that a link points to a downloadable file? The only way todo that is to filter file extensions, but that won't work with a lot of redirection scripts and tracking scripts as they don't use the file extension anyway.
    On the proxy php script, the $url holds the url the user would next see or what page the user wants to navigate to either by typing the url in their browser or by clicking a link.
    I'll just write a condition that the page should not be fetched if the $url value holds a url that has downloading extensions or video file extensions or software extensions (.rar, .exe, .mp3, .mp4).
    No good just replacing (str_replace) the extensions on the links present on the proxied pages as user can just type the original url on his browser and hit the ENTER.
    Meaning, if the original page listed a download link like this:

    <a href="http://downloads.com/softwares/catacaustic/anonymous_surfing_browser.exe"></a>

    to this:

    <a href="http://my-proxy.com/proxify?http://downloads.com/softwares/catacaustic/anonymous_surfing_browser.****"></a>

    then clicking that link wont work alright. But all he has to do to by-pass my proxies filter is to type something like the following on his browser and hit the ENTER button for my proxy to start downloading it:

    http://downloads.com/softwares/catacaustic/anonymous_surfing_browser.exe]301 Moved Permanently[/url]

    So you see mate, I am aware of this loophole and have already started taking measures against it. You see, I ain't dumb like you think I am.
    Yes, I never ran a proxy. But, I can always figure things out and take precautions.
    Yes, I never drove a truck. But, if I can drive a car then I can always figure a little out on how to manage the truck. I'm not totally in the dark here like a total non-driver would be in.
    So, you still think I am going to be taken for a ride by the criminals and find myself drowning in deep waters ?


    Originally Posted by UniqueIdeaMan
    6. Add php function(s) in the proxy script to block audio/video streaming. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.
    Originally Posted by Catacaustic
    Again, how would you know that something is a stream? Also, there's a whole lot of aidio and video streaming that's not illegal. Are you going to block that too? (hint, you should, or you'll be paying millions for bandwidth. Think about what will happen when someone starts using your service to watch high-res YouTube videos for 3-4 hours a day)
    If I learn the file extension for streaming then I should manage it like I mentioned previously how I would manage preventing downloads.
    I believe the streaming is not really a LIVE thingy but a file of some sort that immediately records and plays on the page. I'm guessing right or have come close, right ? If not, then atleast the guess was a good one. Right ? Now you see how broad my thinking is ? I try figuring things out and most of the time I manage to figure things out promptly.


    Originally Posted by UniqueIdeaMan
    7. Add php function(s) in the proxy script to block uploads, block file downloads and prevent streaming. That should prevent anyone downloading or uploading or viewing/playing any files related to illegal stuffs.
    Originally Posted by Catacaustic
    How would blocking uploads block anything illegal?
    I would try blocking uploading altogether so not even legal stuffs get uploaded.
    Now, which php function deals with the uploading (file sending protocol) ? Need to add it on the Mini Proxy to prevent uploadings.


    Originally Posted by UniqueIdeaMan
    Q1. So, what is that php function that blocks downloads ? That should prevent anyone uploading malware/viruses, etc. using my proxy ?
    Q2. And, what is that php function that blocks audio/video streaming (downloading) ? That should prevent anyone uploading malware/viruses, etc. using my proxy ?
    Q3. And, what is that php function that blocks uploads ? That should prevent anyone uploading malware/viruses, etc. using my proxy.
    Q4. And, what is that php function that blocks audio/video streaming (uploading) ? That should prevent anyone uploading malware/viruses, etc. using my proxy.
    Q5. And, what is that php function that prevents the user's browser from playing any audio/video files on a website ? Eg. Prevent playing youtube vids, vimeo vids, metacafe vids, clickbank vids, etc. ?
    Originally Posted by Catacaustic
    PHP doesn't block things like that. I can write something yourself that can check some things, but you'd need to do that yourself to fit in with your system.
    Ok. Thanks. Go ahead and be my guest. Newbies would learn from your code snippet. If you can then try writing a snippet to prevent the downloading too.
    I'm guessing you are gonna have to write code that deals with http sending data (uploading) and http receiving data (downloading). Right ? But, you're gonna have to write something that can differentiate between what kind of data is coming (text, streaming audio, streaming video, audio file, video file, program/application file (.exe), etc.). Maybe, the extensions list be handy for this ?


    Originally Posted by UniqueIdeaMan
    Q6. And, what is that php function that records bandwidth usage (uploads & downloads and audio/video streaming) ? I might aswell give each account just enough data limit for them to browse text pages but not enough limit to view or listen to audio/video pages (like youtube vid pages).
    Originally Posted by Catacaustic
    There arent any. You'd need to keep the bandwidth logs on the server and parse them to get all of that information. As a warning, that job is a whole lot harder than it sounds.
    Not really. I'll explode & implode. Whenever a page loads on the user's screen, I'll dump the whole page's html to a new variable $html and then break-up all words like so:

    c
    a
    t
    a
    c
    a
    u
    s
    t
    i
    c

    i
    s

    n
    i
    c
    e

    g
    u
    y

    and then count how many lines of text exist in the variable to count how many chars (bytes or 8 bits) exist in the page. Once the chars count goes over the limit in a session then the user's acc would be suspended until the day/month is over.
    See, I'm a guy who can figure a thing or 2 out on the spot on how to proceed. Agree ? Be honest.


    Originally Posted by UniqueIdeaMan
    Q7. Any other features to add to prevent anyone abusing my public proxy service ? If so, which php functions should I use to add them ?
    Originally Posted by Catacaustic
    Nope. You'll be hacked in a month anyway, or you'll be shut down for all of the bad links that you're making available, so there's not much point trying to build in anything extra.

    What it all comes down to is this:

    You have got 0 ide aon how to run a proxy, you don't know how to control it, and you don't understand the issues behind it or how anything works, so you are putting yourself in for anything from a bad site to getting shut down qickly, to getting sued for scraping other sites content, and possibly even arrested for whatever illegal stuff will end up happening through your system. Unless you can get some really great legal advice that says that it's all OK, drop this idea now because the pittiance of money that you'll make from it won't even pay for an initial consultation from the lawyer that you will need.
    So, how come public proxies (anonymouse.org, etc.) don't get hacked then ? Nor get sued for their proxy creating copies of webpages (scraping content) when proxifying them ?
    I don't think scraping content is illegal. If it was, bots (.exe, .php) won't exist. Google Bot won't exist. I build .exe bots and at first I was worried the searchengines would sue me for scraping their links but bot builders assured me that scraping has not got anyone sued. Worst come the bot builder will get a court order to cease. But very rarely.
    The fussy website who fusses over webpage content getting scraped always have the option to add noindex robot tags on his page. That is what the bot builder's lawyer would argue in court. He can also argue that the website had a choice to add "no proxifying" tag. But, he did not.
    Mmm. I just invented a tag! The "no proxifying tag". Alternatively, I can give websites to opt-out from being proxied. This is where I will black-list them on my proxy. Ok ? Good idea ?
    Last edited by UniqueIdeaMan; November 8th, 2017 at 03:34 PM.
  24. #13
  25. Code Monkey V. 0.9
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2005
    Location
    A Land Down Under
    Posts
    2,331
    Rep Power
    2063
    Originally Posted by UniqueIdeaMan
    Q1. What do you mean by "landing pages for the SPAM" ?
    What I mean is not sending out emails, but hosting the URL's that the SPAM'ers use in the emails. Your idea is exactly what they need... Something to track all of the links that they want to promote, so you're working in with them.

    Originally Posted by UniqueIdeaMan
    I can't simply reveal all my ideas here in public if I want others to not copy my idea and launch the program before I can. Let's just say I have 4 groups of people all over the world (a new industry) who want to publicise what they are viewing for commercial, learning, teaching or shopping purposes. When I say "commercial", I mean they would earn money when others view their browsing history. That is their incentive to publicise what they are viewing. They'd be in control to what they publicise and what they don't. They can delete the links later that they publicised earlier on auto mode. In short, just imagine 4 groups of people all over the globe have very good reasons to publicise from time to time what they are viewing.
    Now, I need to cater to their needs. I first wanted to provide a top & bottom framed page (their homepage after they log-in to their account) where they view websites on the bottom frame while the top frame logs the pages they are viewing. That way, there would be no need for my server to fetch pages for them and there would be no risk of anyone using my server to upload and/or download anything malicious. But, html 5 has deprecated the frame & iframe. And so, I was stuck on how to track & log what others browse for their browsing to be made public like they want. At the end, I realised a web proxy has the capacity to log proxified pages. Mini Proxy (Gpl) does not log proxified pages and so I added my own. Built my own account reg, acc login, acc homepage, acc logout pages. Now only logged-in users can use the Mini Proxy (public proxy).
    Now, before I open the service to the public, I just want to make sure no one would abuse the service. I know that, if I provide any form of anonymity feature then that would attract exploiters. Therefore, best I don't promote my service as a proxy but a "publicising service" instead. That way, the good public (who are privacy conscious) and the bad public (anonymous criminals) don't find any interest in it but those 4 concerned parties only. I also need to take proper measures so copy cats don't try abusing my servers to get a competition out of business.
    I can build a system to do that and more without using any sort of proxy. I'd build something where users can just enter URL's and save them, then people cna see what URL's are there. No proxy, no messing around, no issues, no worries and a whole lot easier to administrate. Your way of doing this adds noise (way to many URL's for anyone to actually bother with), complexity (as noted by you not being able to get the proxy working after this long), and potential issues. Other services wouldn't.

    I know you're going to say "that's not what my system does" but in reality yes it is. It's all about sharing, People using your system are just sharing a whole lot more, so it's a lot more noise for people to wade through to see anything that they might want to see.

    Another thing to remember, there's already a few big systems where uses can do this. Twitter, Facebook, etc. All offer link sharing, monetising and advertising. What you offer is harder to use, and offers less.

    Originally Posted by UniqueIdeaMan
    What do you mean ? I don't understand your hint. The whole purpose of identifying the user (account holder) via his domain name is so that no user submits his free email address (hotmail, etc.) but an email belonging to his domain name so he knows his internet activities are getting logged under his domain name and if he tries abusing the service then the law will trace him via his domain registrar and prosecute him.
    The email address verification would be done by the script. Not by me manually.
    Email addresses can be hijacked, re-directed, hacked, etc. Even if it's all legit, how would you track down someone that has domain privacy enabled so you can't see their details anywhere?

    Originally Posted by UniqueIdeaMan
    I don't need to. Look at the answers here:
    https://stackoverflow.com/questions/...in-other-words
    So, now you agree I took care of this pitfall you are fearing ?
    That doesn't answer a lot of points. Here's a few examples:

    **** - A rooster.
    **** - A gentlemans first name.

    Get the idea? Filtering doesn't take into account the context of the message. You could filter out things that are perfectly harmless.

    Originally Posted by UniqueIdeaMan
    On the proxy php script, the $url holds the url the user would next see or what page the user wants to navigate to either by typing the url in their browser or by clicking a link.
    I'll just write a condition that the page should not be fetched if the $url value holds a url that has downloading extensions or video file extensions or software extensions (.rar, .exe, .mp3, .mp4).
    No good just replacing (str_replace) the extensions on the links present on the proxied pages as user can just type the original url on his browser and hit the ENTER.
    Meaning, if the original page listed a download link like this:

    <a rel="nofollow" href="http://downloads.com/softwares/catacaustic/anonymous_surfing_browser.exe"></a>

    to this:

    <a rel="nofollow" href="http://my-proxy.com/proxify?http://downloads.com/softwares/catacaustic/anonymous_surfing_browser.****"></a>

    then clicking that link wont work alright. But all he has to do to by-pass my proxies filter is to type something like the following on his browser and hit the ENTER button for my proxy to start downloading it:

    http://downloads.com/softwares/catacaustic/anonymous_surfing_browser.exe]301 Moved Permanently[/url]

    So you see mate, I am aware of this loophole and have already started taking measures against it. You see, I ain't dumb like you think I am.
    Yes, I never ran a proxy. But, I can always figure things out and take precautions.
    Yes, I never drove a truck. But, if I can drive a car then I can always figure a little out on how to manage the truck. I'm not totally in the dark here like a total non-driver would be in.
    So, you still think I am going to be taken for a ride by the criminals and find myself drowning in deep waters ?
    What if the downoad link looks like this:

    Example Domain

    What in there tells you that the file is a downloadable file, and not a "standard" HTML/CSS/etc file? What if that link is sharing malware, and because the spammers have signed up to your system using a private domain, they are now sharing malware links through your system? That's just one scenario, but there's a lot more. You haven't thought of any of that at all.

    Originally Posted by UniqueIdeaMan
    I would try blocking uploading altogether so not even legal stuffs get uploaded.
    Now, which php function deals with the uploading (file sending protocol) ? Need to add it on the Mini Proxy to prevent uploadings.
    All uploading is done via HTTP And your server before it gets to PHP, so as I said before, there's no PHP function to guard against that. You can do checks after it's all done, but that's all.

    Originally Posted by UniqueIdeaMan
    Not really. I'll explode & implode. Whenever a page loads on the user's screen, I'll dump the whole page's html to a new variable $html and then break-up all words like so:

    c
    a
    t
    a
    c
    a
    u
    s
    t
    i
    c

    i
    s

    n
    i
    c
    e

    g
    u
    y

    and then count how many lines of text exist in the variable to count how many chars (bytes or 8 bits) exist in the page. Once the chars count goes over the limit in a session then the user's acc would be suspended until the day/month is over.
    See, I'm a guy who can figure a thing or 2 out on the spot on how to proceed. Agree ? Be honest.
    Terrible. You're going to process every page to get the length when you send it out. Your way of doing it is by far teh least efficient method available. That will add time and complexity to your system again, and it will only work on text-based files. Using your method, how would you calculate the size of a JPG file?

    Originally Posted by UniqueIdeaMan
    So, how come public proxies (anonymouse.org, etc.) don't get hacked. Nor get sued for their proxy creating copies of webpages (scraping content) when proxifying them ?
    They know what they are doing, have spent years doing this, and have hired people that know how the internet works. You don't have any of that, so you're open to all sorts of abuse.

    Originally Posted by UniqueIdeaMan
    I don't think scraping content is illegal. If it was, bots (.exe, .php) won't exist.
    Scraping isn't illegal, but using that for your own commercial use may be depending on the jurisdiction. Laws vary between regions and countries, so youll need to know all of the variations and how they all work.

    Comments on this post

    • UniqueIdeaMan agrees : Good points raised. Some I already knew and am aware of them and some I knew but forgot about them. This post reminded me of what I forgotten or overlooked.
  26. #14
  27. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    346
    Rep Power
    1
    Catacaustic,

    I gave you a REP for your latest post.
    Anyway, so there ain't no php function that can detect the size of an img or audio or video file. On Win OS, we can see each file's size. Php should have had that capacity so we programmers can deal with certain sized files in certain ways with the condition.
    If img file > 1oMB;
    Do this;
    Else do that.

    Catacaustic, I have thought of a workaround to detect audio, video downloads and streamings. I can put a bandwidth counter. So, when a user is on a page and that user's bandwidth counter shows large amounts of bandwidth getting used and he is still on the same url then that means he is on a streaming page or download page downloading audio or video.
    Now, I need to find the bandwidth counting php function. Or maybe, you can build a custom function and share it all over the internet and become the latest php hero ?

    Catacaustic, one thing I don't understand. You use third party computer/server as a proxy to hide your ip from the sites you are visiting and remain anonymous. Your browser requests my proxy to fetch a page (eg. devshed.com). Your browser hands my proxy server your ip so my proxy can know on which direction to send the devshed.com page's html to you. My proxy server makes a request to devshed.com's web server for the devshed.com page by handing my server's ip. Devshed server hands out the devshed.com page's html to my proxy who then forwards it to your browser.
    My proxy is the middle man. My proxy's ip is the middle man ip that devshed's server gets.
    Now, if you run your own proxy by installing it on your home computer (localhost) and use that localhost proxy to browse the net then how is that keeping you anonymous since your proxy server (software) is hosted on your local computer and your local proxy server is using your local computer's ip to fetch the pages you want to surf anonymously ? Does not make sense. What am I missing here. If I can understand that, then a broad horizon maybe opened for my venture in a new light.
    Does anyone know the answer to this ? If you google, then you will see articles upon articles that are teaching you to run your own web or socks proxy on your localhost or home computer (the computer you would use to surf privately) to surf anonymously. How can you remain anonymous like that when you are using no other server in the middle as piggy back ? Strange! Or, maybe the article writers do not know what they are talking about ? Are they all silly billies ? Lol!

    Catacaustic, you say those who run web proxies have yrs of experience etc. I think most of them are amateurs. Even this article says you don't need to know much and I'm getting the hint that I will still be ok:

    "It can get complex under the hood, but you donít need to know too much about how it works to carry out day-to-day tasks."

    https://www.godaddy.com/garage/how-t...-proxy-server/

    We programmers can never be too careful! A layman just goes about and does his thing without taking too much precautions or worrying or fussing over security too much. It's the experts who really fuss over security too much! Lol!
    Last edited by UniqueIdeaMan; December 14th, 2017 at 01:09 PM.
  28. #15
  29. Wiser? Not exactly.
    Devshed God 2nd Plane (6000 - 6499 posts)

    Join Date
    May 2001
    Location
    Bonita Springs, FL
    Posts
    6,126
    Rep Power
    4103
    Since you asked me to chime in UniqueIdeaMan, I skimmed the thread, seems like Catacaustic covered things well.

    Originally Posted by UniqueIdeaMan
    Anyway, so there ain't no php function that can detect the size of an img or audio or video file. On Win OS, we can see each file's size. Php should have had that capacity so we programmers can deal with certain sized files in certain ways with the condition.
    PHP Has a way to get a files size. You can't get the size of a remote file without either downloading it or at the very least making a HEAD request and checking for a Content-Length header though, so it's not going to help you limit the size of files users can download with your proxy.

    You could monitor the size of a download as it comes in and kill the download after a certain limit, but that's not a great user experience.

    Originally Posted by UniqueIdeaMan
    Does anyone know the answer to this ? If you google, then you will see articles upon articles that are teaching you to run your own web or socks proxy on your localhost or home computer (the computer you would use to surf privately) to surf anonymously. How can you remain anonymous like that when you are using no other server in the middle as piggy back ? Strange! Or, maybe the article writers do not know what they are talking about ? Are they all silly billies ? Lol!
    You'd have to link some of the articles your referencing to say for certain whether they are silly or have a reason. There are reasons one might run a local proxy, such as to cache things, perform filtering/scanning, etc. Usually the goal is not to be anonymous though there are still somethings a local proxy could do for that such as stripping out cookies, javascript, user agent data, referral data, etc. Most serious tracking implementations don't care as much about your IP and instead track via cookies based tokens or other data points. IP's are not as unique as some people try to make them out to be.


    Originally Posted by UniqueIdeaMan
    So, how come public proxies (anonymouse.org, etc.) don't get hacked then ? Nor get sued for their proxy creating copies of webpages (scraping content) when proxifying them ?
    How you do know they haven't? Tons of places get hacked and either don't realize it or don't tell anyone that it happened.

    Originally Posted by UniqueIdeaMan
    but bot builders assured me that scraping has not got anyone sued.
    Well that's not true.
    - How I got sued by Facebook
    - Craigslist Files Another Suit against Data Scraper
    - LinkedIn sues anonymous data scrapers

    In the LinkedIn case, it was recently ruled that scraping public data is ok, however it's still a bit early to know for sure what that means in different context. Also, even if scraping the data is ok, how one uses it may not be.

    Originally Posted by UniqueIdeaMan
    Catacaustic, you say those who run web proxies have yrs of experience etc. I think most of them are amateurs. Even this article says you don't need to know much and I'm getting the hint that I will still be ok:
    The write of that article isn't exactly some expert in the world of computer security, or even web development; he's a wordpress guy and a writer. That article is also about setting up your own personal proxy, not something public for the world to use. There's plenty of information on the web that skips over potential security issues when the goal is to make something for your own personal/private use rather than for public consumption. For example I've made plenty of posts here and on other forums showing how to do things that one really ought to not do in a production application but are likely just fine for getting some one-off/personal script up and running.

    Originally Posted by UniqueIdeaMan
    But, html 5 has deprecated the frame & iframe.
    iframe's are still perfectly fine. You won't be able to get much info from them due to the same-origin policy though, and sites can block being loaded in a frame
    Recycle your old CD's



    If I helped you out, show some love with some reputation, or tip with Bitcoins to 1N645HfYf63UbcvxajLKiSKpYHAq2Zxud

IMN logo majestic logo threadwatch logo seochat tools logo