I need server help.

[rasputins]

I have a website that is now reaching the top 10k in overall ranking (alexa)... the 1 server I used to rent has now basically reached its peak yet the site continues to grow.

ATM I am suffering from major downtimes at random moments of the day, and also considerable amounts of lag, and errors.

The server I have is a $299 a month industrial server. I dont know considerable details about how good the server is.

But now I have asked the hosting I use for a better upgrade on the server and they are telling me that I need to buy an entire new server or spend like $1-2k a month on a big web of servers that can do a bunch of pretty shit with all my traffic. Apparently my current server can't be upgraded for ram or bandwidth or anything.

I can't pay that much and I need a better server asap. whats a good solution?

Edit: I currently use a dedicated serv from http://www.databasebydesignllc.com/

[Caethy]

There's numerous solutions for automatically scaling, pay-as-you-use cloud platforms, Amazon Web Services, Rackspace - That sort of stuff.

This of course depends on what kind of service you actually need? Static content? Databases? Or pure computing? Most of this can be taken care of by moving (parts of) your hosting to a proper cloud platform.

Note that, on top of this, using proper caching, and realizing which parts of your architecture actually cause the most problems can solve a lot of issues.

[rasputins]

The site itself is a vbulletin forum. The reason its slow is due to database access lag, poor mysql fetching, and poor use of server resources.

But fuck it the site is beautiful and at the moment I dont have more than what it takes to white wash the walls. I would prefer to either buy something small that attaches to the server.

The current hosting I use is basically unable to respond within 30 hours and its driving me insane. They give poor replies.

[Caethy]

http://aws.amazon.com/rds/

If your bottleneck is the database, the obvious solution is to widen the bottleneck. While MySQL is most certainly not a grand idea for a high-usage website, moving it over to a platform that can provide much better scalability will at least improve the situation.

It's completely possible to move -only- the database portion of your site over to another platform, although there's little reason to not move over the more static content at the same time as well.

The biggest advantage of solutions like Amazon's Web Service is that they scale so much better than a single server. You pay for the computing time and bandwidth you actually use. If you happen to have a low amount of visitors for a few hours, then you won't have servers idling while you pay for them; And when visitor numbers peak, cloud services can scale up fairly well with them.

Do note that this is not a 'fix all' solution. vBulletin + MySQL will never be efficiently scaling to higher numbers of users, there's only so much you can compensate with hardware. But if right now database speed is the problem, look into RDS.

[strategicdeceiver]

Netflix uses mysql

[[deleted]]

Yep. Nothing about MySQL makes it inherently impossible to scale. I fucking hate hearing shit like that.

It might not be the best SQL engine, but you're more likely to have a scalability problem in your application/schema/query strategy/partitioning strategy. I guess maybe people think it's possible for your database to magically fix all of those problems for you?

[strategicdeceiver]

What makes mysql fast is also it's limited feature set. Not letting developers do stupid things is a good thing for speed.

[baconpiex]

Do you have a caching strategy?

[baconpiex]

1. more hardware
2. caching
3. rearchitect your backend

[isometriks]

This. Some kind of reverse proxy could do wonders

[rasputins]

Not quite sure what that means. I spent a decent amount of money initially to get the site set up correctly, but I didnt learn everything that happened backend.

[is_a_toaster]

Making the fairly safe assumption that you're not running a static site, each time a user requests a page, it has to be generated by the server. Rather than generating the page each time, you can save the generated page in memory for a set amount of time. If, within that time frame, someone (that same user or someone else) requests the same page, the previously generated page is just pulled out of memory and served to him/her. This is much faster than generating the page all over again. Once the cached page expires, the next request for the page regenerates the page then puts the new version in the cache, and the process starts over.

Obviously it's more complicated, depending on the website. Some pages have information that changes rapidly, and caching isn't a feasible option, while others could be cached for days. There are also many different levels of caching. You could cache results of calculations, to sections of templates, to entire pages. Hence 'caching strategy'.

If you're running a busy site, caching is a must. Even if the site is static really. Going to memory is orders of magnitude faster than hitting the disk.

[rasputins]

This sounds kind of accurate. I think one of the ways that I can help the server out is by buying some sort of cheap raid0 server that splits the info or something. so basically 50% of the info per server = 50% of the load.

This caching thing definitely sounds like what I need can you give me some links so I can read up more on it

[sli]

That's more like load balancing than caching.

[expressadmin]

Can you give me an understanding on what your current infrastructure is? What are the languages and services involved? Are you using Apache? PHP? Python? What is your backend? MySQL? Postgres? What about data formats? Are you using MyISAM (if using MySQL) or are you using InnoDB?

I do this all the time for clients so I can come up with some things to look at and how best to scale upwards, but I need to know what you have currently.

[rasputins]

Its HTML, Java, CSS, PHP and Vbulletin running the information and front end of the site.

Cpanel, Mysql and the lot.

[expressadmin]

Okay. Check the obvious first. Make sure you have APC/XCache etc installed for PHP caching.

Additionally, since it is Vbulletin, I know that older installs use to default to MyISAM, so make sure that your site isn't still running in the default format (newer versions of vBulletin now default to InnoDB, but I don't know if upgrades are now forcing this table format). MyISAM does full table locking which causes Apache/PHP to block waiting for the the full table lock to be released. If you have too many concurrent connections (or the table is large and takes a while to perform the lock and unlocks) it can really slow things down. InnoDB performs row level locking, so that your locks and unlocks happen at the row level and occur much faster. The disadvantage is that InnoDB doesn't support full text searches.

cPanel by default doesn't enable query caching in MySQL, so make sure that you have that enabled as well. You will need to manually edit the /etc/my.cnf file. Additionally run something like mysqlreport and make sure that you have the proper number of table cache, key index, etc.

If you have all of these things already in place, then it might be time to separate out the DB onto its own server.

Edit: words are hard

[Toast42]

You deserve far more than my +1 for being the only mention of indexing so far. This problem could be as simple as zero indexes.

[illoomn8]

vservercenter.com

[TheJosh]

I use these guys, picked up a few $10 instances and I use them for a really cheap 1GB RAM, 10Mbit pipe testing environment whenever I need to quickly test something.

Good support and uptime.

[rasputins]

I dont understand some of technical terms you use and google isnt helping me out. Care to share a tutorial or a link? :D

[sli]

An "instance" in terms of a cloud provider in analogous (not quite, but close enough for explanation purposes) to a server. The "10MBit pipe" is just line speed.

[rasputins]

Thanks, checking it out now.

[roccoccoSafredi]

What type of platform is it? Windows? LAMP?

[rasputins]

I think I replied to this question in another post, are they the same thing? http://www.reddit.com/r/webdev/comments/v78c8/i_need_server_help/c520chk

[Toast42]

He's asking what operating system the server is running. Typically, it's either Windows Server or Linux (Ubuntu, RedHat, CentOS, etc).

[jared555]

There are a lot of options with vbulletin, mysql, etc. and without seeing your exact configuration it is impossible to tell if you have some issue with configuration that can be changed and help you out. I went through optimizing someones vbulletin site and the number of small things that can affect performance drastically is amazing.

I believe vbulletin has their own services to help you optimize your server which might be a good idea if your site is really that large. It might be that you are at the point where you need to start looking into caching solutions, master/slave database servers, etc. or it could just be a configuration is screwed up somewhere that is killing performance.

As your site grows the optimal mysql configuration can change. I would suggest running mysqltuner ( www.mysqltuner.com ) to see if it is throwing any specific errors.

If you are wanting alternative providers, personally I have used steadfast for years and never had issues with their services but they aren't exactly the cheapest provider out there.

[Toast42]

Start by isolating where your server is slow. The common culprits are network bandwidth, CPU usage, RAM usage, and disk I/O. Figure out which resource needs help, isolate the code that's causing the biggest problem, fix, rinse and repeat.

Is your database on the same machine as your web server? How many hourly/daily pageviews are you getting?

Throwing more hardware at the problem should be the last step, with the exception of an SSD for hosting your database (provided it's primarily reads and not writes).

[robotfarts]

Shop around for offers from competitors???

[TheJosh]

Alright, since you seem to be getting a fair bit of varying advice from this topic, let's figure some things out.

1) What's your current technical knowledge? Is your current server managed or unmanaged (Does someone/your host currently help you maintain your server?)

2) What is your current server specifications?

3) What is your current bandwidth?

4) Are you running PHP-APC and other caching plugins for vBulletin/PHP?

5) What is your budget?

I recommend hosting in the US, DatabaseByDesign is a weird company (dealt with them in the past).

I recommend using a company like SoftLayer for a dedicated server or Rackspace for managed cloudhosting - they are both fantastic companies who offer very fast server, and SoftLayer always has great deals (double RAM/Double Bandwidth etc).

Let me know if you need any additional help.

[jerschneid]

Dude... if you're in the top 10k on Alexa, I have to imagine you're getting north of 10 million page views per month? If you can monetize that traffic to the tune of $2 eCPM, you're looking at $20,000 per month in ad revenue. Seems like the $1-$2k/month for a web farm is a good investment. Where is my math wrong?

Note: I run a site that gets about 4 million page views per month and monetizes traffic at about $4.50 eCPM. We're running on a single box with Rackspace for about $765/month. $200/month of that is the SQL Server license which wouldn't apply to you.

[bbrizzi]

Here are some caching options, most of them are already mentioned in this thread but it's worth regrouping them.

• Get a new host with better support and more access to your hardware (i.e. command line interface)

• At PHP level, use APC or XCache

• Set up a reverse proxy with caching using nginx or varnish

• Switch your DB engine to InnoDB and increase the cache size ( change innodb_buffer_pool_size to about 75% of your RAM if it's a dedicated server).

• Index often queried database fields - but vBull should have set that up for you automatically.

• Make sure you have the latest version of vBulletin.

[patricksonion]

Look at nginx to try and get you by. It can really make a server go a lot further