#1
  1. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,664
    Rep Power
    171

    Too many connections to database.


    Hi;

    How can I test the website under extremely big traffic?

    Im running a natinal tv ad soon and I want to make sure I dont get 'too many connections'.

    Thanks
  2. #2
  3. Did you steal it?
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    14,074
    Rep Power
    9398
  4. #3
  5. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,661
    Rep Power
    4123
    for a brute force naive approach I use apache benchmark, which is a command line tool installed with apache

    I spin up a new cloud server, install apache and then hit my sites and apps until they break.

    The important thing about load testing is to run it from a different server - this way the server is only processing requests and your results are not skewed by the software actually requesting the load.

    If you've got any profiling running then you can see any bottlenecks in your code.

    If you have mysql workbench you can watch your database performance.

    Your most used (and most limited) resource is RAM, so configuring how this is divided up between your applications is important, because as soon as you run out your site/app will grind to a halt as the server starts "swapping"

    A site allowing 100 concurrent users with correctly configured RAM distribution will still perform better under a high load than a server with allowing 1000 with mis configured RAM. This is because connections are, in a way, queued so the faster you can handle a request, the more requests you can deal with per second (ie a rate). If you can handle 100 in 0.1 seconds thats 1000 per second, but if 150 requests eats all your RAM then the request time for new requests will start to go into the seconds (or tens of seconds).

    Here's a rough guide I work to:

    Total RAM - OS use = available RAM
    Put the database into ram (see other posts on how to do this)

    available RAM - database size = RAM for PHP

    find out average size of PHP request. This could be up to 128Mb (I'd worry). Wordpress runs between 20Mb and 40 Mb. I try to code for <2Mb

    RAM for PHP / PHP request size = number of concurrent connections

    Theoretically, at full load your server would always be operating in RAM - so quickly. As soon as this overflows into using hard disk space (called swapping) then the server slows down because the RAM performs several orders of magnitude faster then your HDD. Even if you chose solid state HDDs (SSD) these are still one or two orders of magnitude slower then RAM
    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  6. #4
  7. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,664
    Rep Power
    171
    Hi;

    Northie in your database class what things can cause "too many database connections" error and how can I fix it?

    Thank you
  8. #5
  9. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,664
    Rep Power
    171
    Northie;

    I have the set up of Mysql Workbench and I can see the real time status, queries, etc.

    I just wonder how I can make ~500 requests at the same time and simulate the heavy traffic.

    Thanks
  10. #6
  11. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,661
    Rep Power
    4123
    Have a look at the mysql docs here and then see what your connections are set to, and alter if if needs be.

    We've already given you answers on how to simulate multiple requests (Seige and ab) - but don't DOS attack your live site!
    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  12. #7
  13. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,661
    Rep Power
    4123
    I've just turned off request caching for my api and tried to hit my site with 100 concurrent connections repeatedly until I had completed a few thousand requests....MySQL reported no more than 31 connections. Basically, my api is so fast, the reporting tools can't keep up with the number of opens/closes to the database. I'm getting 130 - 150 requests per second handled for new, uncached requests and 260 - 315 requests per second for cached requests. I'm not using more then 30% of my 512Mb RAM....so I'm wondering what to look at next in terms of optimisation (or just move on to building the rest of the app!)
    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  14. #8
  15. A Change of Season
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2004
    Location
    Next Door
    Posts
    2,664
    Rep Power
    171
    Thanks guys;

    Siedge wasn't east to instal. I instead used Jmeter which is absolutely great. I use it along with Mysql Workbench and monitor both screens.

    What I found was first: A query that was freezing the site.

    And also max_connections. I seem to be limited to ~ 100 connections. I found some code that I can change and I believe will fix the problem but I am waiting for reply on the post before I upload the conf file, hopefully you can confrim my solution and I apply changes.

    Thank you.

IMN logo majestic logo threadwatch logo seochat tools logo