Forum temporarily disabled, bandwidth exceeded, website dying, er, “help?”

July 3rd, 2007 by Ben Goldacre in bad science | 108 Comments »

Sorry about the very long outage, my host suddenly and without any announcement simply pulled the plug on all CGI (geek stuff) because I was apparently putting too great a strain on their servers, and I’ve had to switch the forum off (at the very least) until I get it sorted. It turns out that I was the hog making everything run slowly: who’d have thought? This blog is back online but only on borrowed time now I’m afraid, to give me a chance to upgrade to WordPress 2.2.1 and see if that will help reduce the CGI load for a bit. I’ve barely got time to do even that right now.

If anyone could offer help that would be amazing, I need to upgrade WordPress and phpBB. I really can’t pay for web skills at the moment – must start vitamin pill company instead of giving talks for free in schools and losing train ticket receipts – but I can give love, kudos/publicity, and I know some pretty girls. Might be better to email instead of posting below as the site will probably go down again soon, maybe for quite a while, so ben@badscience.net is the best.

Also unfortunately – with the best will in the world – I guess I maybe now ought to move to a bigger host to speed everything up, and most importantly remove any chance of this kind of thing happening again (the hours wasted already…) so it would be good to get opinions on what hosting can handle a min of 12,000 visitors a day to a mySQL intensive WordPress and phpBB installation, and run really quickly? Ideally somewhere a step up from my current ten dollar arrangement but a bit short of the proper big money for a dedicated server?

Anyway thanks and sorry for the outage, hopefully going to sort it all out for good this time. Thank god nobody ever goes here or here eh. Look ma, no adverts. No evil drug company funding. Just a broken website. Looks like the web2.0 cashless volunteer ideas economy may not work if you lack time and 3L33t webskillz.

Sticks in my throat. “Help.” Can’t someone please just become my best mate and hold my hand for the web stuff? I really love this website being here, but I just can’t spend hours sorting it out at the moment.

update: they say if it goes over the mysql limit at all again then it will get cut off, i closed the forums, upgrading wordpress might buy some time but i cant do it myself right now.

to be clear, copied from a scattily written email, if anyone can help right now my needs (as well as getting through punishing outpatients clinic) are:

1. upgrade wordpress and phpbb to latest versions to buy some time with current host, and get myself good back ups

2. identify good medium grade shared host – would rather pay and have certainty etc. i need somewhere better than $10 but ideally not £100 a month dedicated server if can avoid. i want to pay and have reliability and not be beholden. site gets 12,000 visitors a day, 2gb/day, but that will go up quite a lot with the podcasts, interviews, and lectures that are coming v soon. it really hammers the mysql servers, because of being wordpress and phpbb, this is much more important than bandwidth: fast good mysql servers.

3. migrate to new server

4. there is also some really interesting medium term work to be done on interesting modest community building tweaky aspirations, eg get the forum better integrated, but most importantly, take the miniblog, and instead of it just being a tiny feed, making it so that when you click, you go to a discussion page, like the b3ta links board i guess, with the link at the top, so people can follow it from there, or discuss the content, because at the mo there’s great stuff in the links board and could form the focus of some really interesting new geek community building, especially if i can start dishing out logins to other people to post interesting links there too. the stuff in the miniblog gets discussed a bit in the forums, but would be good to have more opps for chatter, since the standard of banter on the board is so high.

5. ideally someone to help me with everyday maintenance of when things cock up etc, or adding amusing tweaks.

ideally there’d be a couple of people to help fix this current problem, and then a dedicated young whippersnapper who fancies the community building challenge of the new miniblog links discussion thing.

cant really offer cash, although some paypal things are coming in today. the site costs me money as a rule, which i’m fine with, although cant carry frightening 40 quid an hour fixing things bills and very tough to find the full day it would take me to sort this with my legendarily poor webskillz as fulltime junior doc and column/blog only hobby, and all these unpaid talks i spend my time doing which i love but where i lose the receipts even for expenses are frighteningly closely booked right now.

so, dont like to be dependent or demanding, but i guess i do spend half my time talking to people about their kids medical problems, notably without charging them 40 pounds an hour for the privilege. can offer adulation and free glory, and very good karma. and at some stage i’ll leave my low pay jobs, qualify fully as a proper grown up major doctor, accept an offer from another newspaper or something, and be rich enough to pay through nose. that day will indeed come. ideally i’d become best mates with a really motivated geek with lots of spare time looking to build a portfolio and cv from a well visited site, and if cash ever comes in from some related big web project (working on it) then you’d be first port of call. trreading fine line between grovelling, whining and carroting here.

del.icio.us Digg it reddit Google StumbleUpon Slashdot It!

108 Responses



  1. Alan Cann said,

    July 3, 2007 at 6:37 am

    Ben: Siteground:
    https://www.siteground.com/

  2. Ithika said,

    July 3, 2007 at 9:17 am

    I promise that the next thing I buy from Amazon will be through your store! Promise! I’ve been eyeing up the Sokal/Bricmont book, as it sounds like a good laugh and/or something to make me properly angry.

  3. Despard said,

    July 3, 2007 at 10:17 am

    Sniffhost are pretty cool. I recommend them highly.

    http://www.sniffhost.com/

  4. Lave said,

    July 3, 2007 at 10:25 am

    hhˇ tjkj
    Damn sorry Ben. Good luck – but I know nothing I can help with. Apart from that I’ve made this thread:

    http://skeptobot.blogspot.com/2007/07/bad-science-emergency-chat-thread.html

    Where people from the forum can leave messages and chat in the mean time. And if anyone finds a better solution I’ll link to it from there. And includes and any updates and so on.

    And all amazon purchases will now go through you.

  5. Nickynockynoonoo said,

    July 3, 2007 at 10:38 am

    A small donation will be coming your way.

    Come on guys! A little from everyone for this excellent site could possibly give Ben a dedicated server. (I haven’t a clue how much is needed)

    I really missed my badscience fix yesterday.

  6. troutboy said,

    July 3, 2007 at 10:40 am

    Just made a donation now. The world needs websites like this.

  7. tom1 said,

    July 3, 2007 at 10:59 am

    Ben,

    Time for the annual monitoring of the servers was it? How could they not see this coming? Very odd. There have definately been intermittent problems accessing the site for more than a week. I had assumed that Panorama (dross again) being on last nighth had caused you to be swamped.

    The big thing here seems to be that they aren’t focused on professional blogs/forums (do they know you write for a national newspaper?). In my limited experience of these things, I’d have thought the thing to look for was 24/7 phone support and possibly some kind of monitoring. All of that costs money though. I rent a dedicated server, so I can’t really recommend a webhosting company.

    If at all possible you want to stick away from anything bespoke supported by somebody who you don’t have a contract with. Also, and I guess this is a minor point, but whoever helped you would probably have access to your user database. It’s very unlikely, but it would be embarassing if the Bad Science user database ended up on the web. Ideally persuade your current hosts to put up with you until you find host you’re happy with. Can’t the Guardian advise on this (surely they’re into the whole new media thing)?

    In the real world I build/support/fix a variety of systems for a bank. I’ve turned my hand to webservers and php in the past. You’d be much better off getting somebody who played with Apache? and php every day though.

    If there really is a risk of them shutting you down again, I’m sure myself, or someone else could set you up with a backup server so it would just be a case of repointing the domain name. How much of an emergency is this? Given how helpful and randomly draconian they have been so far, make sure you have a good backup of you mysql database and anything else that it would be painful to lose sitting somewhere other than with your hosting company.

    Regards,

    Tom

  8. Diamond said,

    July 3, 2007 at 11:21 am

    Ben,

    Can I ask how much you spend/can spend on hosting because I might be able to help host you on my dedicated server.

  9. Nero said,

    July 3, 2007 at 11:30 am

    Ben,

    I hope you don’t mind but I posted about your predicament on the JREF forum.

    http://forums.randi.org/showthread.php?t=86374

    Some useful suggestions have been posted, that you may want to follow up.

  10. Lave said,

    July 3, 2007 at 11:33 am

    Just gave you a tenner for website costs. Come on cheapskates, I can’t cope without Bad Science.

  11. gnu said,

    July 3, 2007 at 11:36 am

    Sorry, have no idea how t’interweb works and therefore no helpful suggestions, have some money instead.

  12. jackpt said,

    July 3, 2007 at 11:36 am

    I have also sent you something, hopefully it won’t get eaten by Gmail’s spam filter.

  13. tom1 said,

    July 3, 2007 at 11:37 am

    All,

    when we’re talking about a dedicated server, do we mean a dedicated webserver, or a dedicated linux box that you install Apache on yourself. If it’s the dedicated linux box option, I’m using domaingurus.com. When I chose them they seemed very cheap and their support really good.

    Without looking this up, my impression about cost is:
    Basic webhosting ~£10+
    Virtual Dedicated Server ~£30+
    Dedicated Server ~£60+

    Is the dedicated/not dedicated side of it important? What we need is a webhosting company that is OK with the load and has a more helpful attitude.

    Tom

  14. tom1 said,

    July 3, 2007 at 11:51 am

    Ben,

    I think we need clarification on just what the problem is being solved here.

    1. Is there a risk the Bad Science will be shut down again (if so how imminent).
    2. Is there a longer term funding problem here? It’s not as if Bad Science requires a server farm.
    3. Is there a long term support/administration problem?
    4. Are you just asking for advice on a good webhosting company?

    Tom

  15. Digital Goldfish said,

    July 3, 2007 at 12:10 pm

    Donated £20, mate.. Hope it helps! We need Bad Science!!

  16. Tristan said,

    July 3, 2007 at 12:39 pm

    Just donated £15. Good luck.

    Now, about those pretty girls you said you know….

  17. Tristan said,

    July 3, 2007 at 12:51 pm

    Just bought a couple of books through your amazon site too. Those girls had better be very pretty Ben!

  18. Aspiring Pedant said,

    July 3, 2007 at 1:19 pm

    I’ve added an Ayrton to the fund but I’m afraid I can’t offer any practical assistance.

  19. Andrew Clegg said,

    July 3, 2007 at 1:31 pm

    It seems a bit harsh that your hosts can arbitrarily cut off your CGI scripts because they can’t handle them. After all you’re paying for a service here, and it’s their problem if their servers are pants. Alright, there’s a few users here, but it’s not exactly slashdot!

    The deal on that top link (siteground.com) looks pretty tasty and it does say unlimited MySQL… And has WordPress and phpBB as part of the deal.

    Once you start publishing podcasts, consider seeding them via BitTorrent rather than just having the punters download the files via HTTP. That way, if more than one user is downloading at once, they’ll share the load between them and take the heat off your bandwidth… Just a random idea.

    Andrew.

  20. nic0 said,

    July 3, 2007 at 1:38 pm

    Can anyone confirm the paypal email address to mail to for donations? ben@badscience.net is OK? The button on http://www.badscience.net/?page_id=376 seems to be throwing up an error.

  21. tom1 said,

    July 3, 2007 at 1:45 pm

    At the moment there seem to be two urgent jobs.

    1. Get the forum properly backed up.
    2. Upgrade WordPress and phpBB.

    A big chunk of my day job is backup, so if nobody else is keen, I offer my services for that.

    As for WordPress/phpBB. Does any body have any experience of them? If not, it shouldn’t be too difficult to figure out.

    The least risky way of addressing all of this is for the webhosting company to be persuaded to be slightly more helpful. There is definately more that they could do. It’s their fault for letting it get to this stage after all.

    Tom

  22. Nickynockynoonoo said,

    July 3, 2007 at 1:46 pm

    The link to donate is not working. Can I just do it through paypal to badscience.net? I’m not really familiar with this stuff.

  23. hairnet said,

    July 3, 2007 at 1:51 pm

    Time to cough up people, well done to all who have already done so.

  24. Monkey Doctor said,

    July 3, 2007 at 2:01 pm

    Can’t offer any help transferring, but as alternative hosts go I cannot reccomend register1 (www.register1.net) highly enough. Reasonably priced but more importantly really good service. £100/year will get you 200GB / month traffic and more space than you’ll ever need. I use them for my 2 sites and they have been excellent.

  25. nic0 said,

    July 3, 2007 at 2:02 pm

    To those struggling with the paypal link on the “Support Badscience” page, the imagelink on this page at the bottom of the post is working :)

  26. Lave said,

    July 3, 2007 at 2:03 pm

    I’ve just checked my reciept for giving ben some mulah.

    It says:

    “This email confirms that you have paid bens@goldacre.net … using pay pal”

    So I think the address you want to use is bens@goldacre.net with paypal.

    The button on this page seems to work though.

  27. davehat said,

    July 3, 2007 at 2:09 pm

    Hi Ben. I just emailed you – a mate of mine’s company may be able to help you out…

    I have to echo the recommendations above to make sure your databases are backed up.

    I’m off to have a look down the back of the sofa to see if I can put owt in your tip jar…

  28. Monkey Doctor said,

    July 3, 2007 at 2:10 pm

    Actually Ben, having an attack of change of mind. Email me… I can at least help get the Forum on to a new host, esp if you go with my recommended guys.

  29. BrickWall said,

    July 3, 2007 at 2:37 pm

    Tempting as the offer of working for you is Ben I’m afraid my webskills don’t even extend to having ever heard of WordPress or mySQL before and not having a clue what most of your statement of requirements actually means.

    Mind you not being qualified or able to do the job doesn’t seem to stop people applying to jobs here (can be quite amusing sifting through application forms to discover people listing Sudoko as evidence of numerical skills!)

    Thrown a tenner your way, sorry failed yet again to wing lottery or else I’d help out more generously.

  30. seenoevil said,

    July 3, 2007 at 2:39 pm

    Sent some spondo’s, can’t help with anything else though. Hoping this issue gets resolved soon.

  31. Nickynockynoonoo said,

    July 3, 2007 at 2:50 pm

    Thanks Nic0, that link works fine.

  32. tom1 said,

    July 3, 2007 at 2:59 pm

    There are some articles on backing up WordPress at:

    There’s a step by step techie one…
    http://codex.wordpress.org/Backing_Up_Your_Database

    And a best practice/overview type one…..
    http://codex.wordpress.org/WordPress_Backups

    Assuming this is the same WordPress that Ben’s using, it doesn’t look too tough at all.

    Again, if there aren’t decent backups of the site somewhere off the webserver, there really should be. Particularly before anything else gets upgraded.

    I’ll see if I can dig out anything on upgrading WordPress and phpBB.

    Tom

  33. SpallationFiend said,

    July 3, 2007 at 3:00 pm

    Likewise, have sent a few pennies your way.
    Christ, don’t let this place fold up; I don’t know what I’d do during my working hours otherwise.

  34. tom1 said,

    July 3, 2007 at 3:31 pm

    There are upgrade instructions for WordPress here:

    http://codex.wordpress.org/Upgrading_WordPress

  35. tom1 said,

    July 3, 2007 at 3:38 pm

    If the following documentation is to be believed, upgrading phpbb shouldn’t be too bad either:

    http://www.phpbb.com/support/documentation/3.0/

  36. FlammableFlower said,

    July 3, 2007 at 3:42 pm

    What’s the best way to donate without going through paypal? (Have yet to get to grips with that)

    Best of luck!

  37. DavidJ said,

    July 3, 2007 at 4:17 pm

    Small donation been sent. Keep going.

  38. Tristan said,

    July 3, 2007 at 4:25 pm

    Well, seeing the number of donations people are giving I wonder if we might start seeing similar “problems” with the site every month in the week leading up to Ben’s pay day!

    ;-)

  39. tom1 said,

    July 3, 2007 at 4:40 pm

    Anybody else got any thoughts on backing the site up, just in case? Since at the moment we don’t have any WordPress gurus the upgrade is probably the risk here.

    It’s probably overkill, but I guess we could back the content of the site up independently of the database and WordPress. I vaguely recall being able to do a recursive wget…?

  40. Stuu said,

    July 3, 2007 at 4:50 pm

    tom1: Backing the site up is very simple. Download all the files via ftp, and, assuming Ben’s MySQL is handled through something along the lines of phpMyAdmin, you can dump the whole database to a file which can then be downloaded.

    Upgrading WordPress is basically downloading the latest version, then uploading it to your own site, with a few safety measures here and there.

  41. roymondo said,

    July 3, 2007 at 4:52 pm

    Useless with techie stuff so sent a tenner. Please don’t leave The Guardian! That would leave Charlie Brooker carrying the paper on his own (it’d still be better than all the others though.)

  42. Stuu said,

    July 3, 2007 at 4:57 pm

    Also, old versions of WordPress/phpBB probably aren’t the root cause of any problem. Both systems are old enough & well established enough that new versions tend to be more along the lines of security updates, with only small optimisations in server-hammering, if at all (That being said, it is still useful to keep them up to date). It’s probably quite plausible that Ben is hitting a bandwidth limit, or something of the sorts.

  43. tom1 said,

    July 3, 2007 at 5:17 pm

    Stuu,

    I agree that the upgrade is unlikely to do a huge amount to solve the problem, unless the webhosting people are aware of a specific issue.

    As you say, ftp would be a good idea also (belt, braces and something else. I skipped the bit in my reasoning where I say that with a wget we wouldn’t need any usernames or passwords from Ben.

    Regards,

    Tom

  44. Stuu said,

    July 3, 2007 at 5:27 pm

    Tom; why do you say you wouldn’t need passwords for wget? A lot of what you’ll be retrieving will be php, so if you access it by HTTP, you’ll just get the generated page, whereas you want the source code. Also, to get in and dump the database, you’ll need a password at some point.

  45. tom1 said,

    July 3, 2007 at 5:27 pm

    Also, is everybody else getting an error message once in a while saying either the username/password are wrong or it isn’t able to access it’s database? I don’t think it is a bandwidth issue. Maybe they’ve limited the number of concurrent connections to the database?

    The other error I see is your common or garden ‘Internal Server Error’.

    Regards,

    Tom

  46. tom1 said,

    July 3, 2007 at 5:32 pm

    Stuu,

    I think we’re talking at cross purposes. Without Ben giving us a login, we’re stuck with what we can retrieve via a web browser. You can recursively pull down the content of the site. Asside from the user database it’s the only thing that matters.

    Again, it’s a little, possibly very, paranoid and I guess I was looking for something productive to do to help. With my sane hat on you’re ftping the data off is the sensible thing to do.

    Tom

  47. MarkE said,

    July 3, 2007 at 5:54 pm

    I’ve just sent some money: should have done it long ago. Your site is essential even for those of us whose last academic science was Biology O Level 35 years ago.

  48. tom1 said,

    July 3, 2007 at 5:57 pm

    Stuu,

    I just had a look at your site. You use WordPress.

    :-)

    Even if it’s only for the purpose of keeping the webhosting guys happy, I’d be in favor of the upgrade. Who knows, maybe there is a bug…?

    If you’re familiar with WordPress (and phpBB?) do you have any thoughts on the upgrade. The skim read I did of the manual gave me the impression it wasn’t far off a download the update, run the script, click next a few times and then done job.

    Tom

  49. zeno said,

    July 3, 2007 at 6:19 pm

    A small contribution is winging its way via Paypal. Keep up the good work.

  50. Stuu said,

    July 3, 2007 at 6:28 pm

    I do use WordPress, and I’m pretty familiar with its innards; shame the content side of things is so woefully underdeveloped on my site.

    Upgrading WordPress is no more than a reupload of the latest version, and then running an update script. I’ve not used phpBB, but from its documentation, the same thing is required. I’ve e-mailed Ben to say that I’ll happily do the updates for him.

  51. Matt J said,

    July 3, 2007 at 6:33 pm

    When you’ve updated WordPress, start using the wp-cache plugin. It means that dynamic pages are only regenerated when they need to be, so it puts less strain on the server.

  52. Dr* T said,

    July 3, 2007 at 7:18 pm

    Likewise, a small contribution is on its way. Alas, I cannot help further as I am an e-fuckwit.

  53. tom1 said,

    July 3, 2007 at 7:32 pm

    Cool, so there’s a few of us already who feel more or less comfortable doing this.

    Stuu, I guess from your last post that you’ll be OK getting the wp-cache plugin mentioned by Matt working?

    It’s just a matter now of how urgently Ben wants this doing, and who’s available. All being well it should be pretty trival.

    Tom

  54. lucy said,

    July 3, 2007 at 8:15 pm

    I’ve been reading Bad Science for a while now and as an ex-employee of an ISP I feel compelled to add my 2p.

    Sadly, the only ‘help’ Ben’s current ISP is likely to offer is an upgrade to a rather expensive dedicated server. Apart from other people’s offers of help with hosting it might be worth looking at Rose Hosting (in the states) and ByteMark (in the UK). It should be possible to put the site on a virtual server (a very good halfway house between shared hosting and a dedicated server), which will be cheaper than a dedicated server, however I’m not sure about the bandwidth allowances for those types of packages. I’m guessing that with the current shared hosting the MySQL server is on a separate machine and generally under quite heavy load – with a virtual server everything will be on the one machine and you should notice a big performance improvement.

    Like everyone else I would advise taking a backup of the site (never ever trust an ISP to have backups) and upgrading PhpBB and WordPress – any software like should be always be kept uptodate for security reasons if nothing else.

  55. Deano said,

    July 3, 2007 at 8:30 pm

    - duly coughed up – probably owe rather more for bandwidth than most.

    Er – popular site = advertising Ben?

  56. marknesbitt said,

    July 3, 2007 at 8:46 pm

    Hvae paypal-ed a tenner – a modest contribution to keeping this voice of reason online. Mark

  57. tom1 said,

    July 3, 2007 at 8:51 pm

    All,

    I’ve been looking at the wp-cache documentation. I get the impression that this should work on downlevel versions of WordPress. Also it’s pretty clear that if wp-cache isn’t running then WordPress isn’t configured right.

    One question that hasn’t been answered so far is, is wp-cache in use at the moment. If not, then that alone might resolve the immediate issue.

    People on the WordPress support page seem to have made the judegement that (11 months ago) wp-cache was more important than security. At that stage it didn’t work with the newer release of WordPress.
    http://wordpress.org/support/topic/80782

    For another day there’s an article at
    http://codex.wordpress.org/High_Traffic_Tips_For_WordPress
    that maybe Ben should skim over. It covers a good chunk of the longer term discussion.

    Tom

    Tom

  58. tom1 said,

    July 3, 2007 at 9:12 pm

    I don’t know if it’s possible to discover if wp-cache is in use externally, but I think if Ben could check for it in the Installed Plugins list of the plugins tab on the WordPress Admin panel we should be able to confirm whether this is the issue.

    It would be helpful to know what version of WordPress (and possibly phpBB) Bad Science is currently on.

    Tom

  59. Stuu said,

    July 3, 2007 at 9:34 pm

    WordPress 1.5.1.3, cache not used… can’t see what version of phpBB, as it’s down.

  60. bad chemist said,

    July 3, 2007 at 9:39 pm

    Donation made, I’ve have only spent it on beer anyway. I have some webhosting knowledge but it’s been thoroughly eclipsed by the previous offers.

  61. tom1 said,

    July 3, 2007 at 9:50 pm

    Stuu,

    Would it be worth trying to get the cache working before worrying about upgrading? It seems plausible that that’s the problem.

    If it was my webserver, I’d be happier mucking about with the cache than running database updates and upgrading all my scripts in a rush.

    Maybe I’m being too conservative?

    Tom

  62. trickcyclist said,

    July 3, 2007 at 10:44 pm

    Yikes! And I thought the science got a bit technical sometimes…
    Small woo-fighting fee heading your way.

  63. Gimpy said,

    July 3, 2007 at 10:54 pm

    I’ve donated some of the housekeeping budget. It’s a good thing it’s summer as the Gimpy household will be relying on candles and biscuit tins for heat, light and cooking for the next month.
    When Big Pharma come through with their payments for your woo-bashing can I get a cut?

  64. manigen said,

    July 3, 2007 at 11:02 pm

    Can’t help with the tech stuff – I only just barely know what php is – but my donation to the cause is made.

  65. tom1 said,

    July 3, 2007 at 11:09 pm

    A quick final thought. If they are still talking about shutting off the database, I’d like to be clear that we would still have enough access to copy it somewhere else. If we aren’t sure about that, a good backup is even more important.

    Sticking the database somewhere else and repointing the website is pretty easy.

    Fixing the underlying issue would be better though.

    Tom

  66. Stuu said,

    July 3, 2007 at 11:28 pm

    Re: Ben’s latest post; I’d be plenty willing to do all the stuff, with the exception of the forum-minifeed integration, as while I have time to burn, unfortunately I don’t quite have that much.

  67. Jut said,

    July 4, 2007 at 1:07 am

    just did a bunch of interweb shopping on your amazoney thingy:)

  68. beebhack said,

    July 4, 2007 at 7:01 am

    Bunce on the way. Will ask my techie mates for their ideas. I have none, about anything.

  69. tom1 said,

    July 4, 2007 at 8:22 am

    Ok,

    so the plan as I see it is:

    1. Back everything up.
    2. Try and get the cache working.
    3. If the cache doesn’t fix it, upgrade WordPress.

    Either we need to write sufficient instructions for Ben to do this, or Ben needs decide when he can do this and make sure one of us is available to help.

    As for the longer term tasks, they look like fun. Stuu, I’ll arm wrestle you for them… ;-)

    Tom

  70. conejo said,

    July 4, 2007 at 8:29 am

    Donation made.

  71. igb said,

    July 4, 2007 at 8:58 am

    I’ve mailed Ben to offer having a look at tuning the MySQL instance to suit the load on it. As someone points out above, it’s possible he doesn’t have his own instance, in which case it’s all a bit moot.

  72. tom1 said,

    July 4, 2007 at 9:46 am

    igb,

    good to have the input of a database guru. I think the thinking at the moment is that the problem is in the way the application is using the database (without caching) rather than the database itself. Any thoughts?

    At the moment I’m assuming that we’re looking for the simplest solution to deal with the immediate issue with the minimum of risk.

    We could probably do with some input from Ben on what he wants to do, when he wants to do it and how much help he needs.

    Tom

  73. FlammableFlower said,

    July 4, 2007 at 9:53 am

    @55, Deano – don’t start that kind of thinking, there’ll be a riot… ;)

    Ben – Look at what you’ve spawned (the masthead here is now looking quite ironic), you’re too damn popular!

    I’m having severe DTs not getting my forum fix….right..off to donate via your Amazon page as I’m one of the Paypal-illiterati so can’t do direct donation.

  74. germslayer said,

    July 4, 2007 at 10:46 am

    donated a tenner of my chocolate fund…

  75. Phage said,

    July 4, 2007 at 1:21 pm

    Ben, have posted your call to some smart people.
    http://www.hulver.com/scoop/story/2007/7/4/81950/98357

  76. JQH said,

    July 4, 2007 at 1:26 pm

    Donated a tenner of my beer fund.

    As a long term measure, a decent forum host is going to cost something to run is i worth making the forum subscription only either when you have reached a post limit (say 30?) or after a certain period of membership (say three months?). That way we might avoid this sort of problem in the future.

    Just a thought. I’m missing my fix and getting a bit twitchy…

  77. tom1 said,

    July 4, 2007 at 1:47 pm

    JQH,

    I think there is a philosophical question here about whether this is to be a lets-invent-radar-with-some-damp-string-and-whatever-I-can-find-in-the-shed type operation, or start inching in the direction of the JREF where they recently spent $5000 on a new server (though there is considerable room for an intermediate position).

    Also, it’s not clear yet whether the issue is Ben’s global popularity having exceeded the capacity of his host, or just some silly configuration issue.

    If we do need investment, I suppose Ben could always look at some kind of PFI model for funding. I am assured that they are extremely prudent.

    Tom

  78. tom1 said,

    July 4, 2007 at 2:03 pm

    All,

    The following website appears to be very responsive and well maintained:

    http://www.drgillianmckeith.com/

    Perhaps Ben could get in touch with the owner for advice. From the way ‘Doctor’ is used repeatedly on the site, I can only assume it must be run by one of Ben’s colleagues.

  79. briantist@work said,

    July 4, 2007 at 2:29 pm

    Ben,

    I’m always happy to help in my spare time – I can install/program/configure/maintain as necessary.

    Anything to keep up the excellent work.

  80. Alex McLean said,

    July 4, 2007 at 10:20 pm

    Perhaps you could create a facebook group, so we have somewhere to look for news / provide help if the site goes down?

  81. Despard said,

    July 4, 2007 at 11:03 pm

    I’ve already found two using a global search for “badscience”. :-)

  82. tom1 said,

    July 4, 2007 at 11:20 pm

    Facebook will be blocked by a lot of firewalls (my work one for example).

  83. SteveNaive said,

    July 4, 2007 at 11:59 pm

    Important stuff – keep it up – donation made!

  84. Ben Goldacre said,

    July 5, 2007 at 12:13 am

    woah

    ok

    can barely get into the site myself

    and busting up with dayjob work

    briefly: got a server sorted i think, positive internet are the new gods, with a great track record of supporting geek political causes (eg they host richard stallman too)

    lengthy grateful post and a serious lifetime of proper noisy public adulation to follow

    i really dont need money, always happy to sub the site within reason, but anything that comes in will go to petty cash for podcast gear, expenses, sundries, and crack cocaine as ever.

    this has been a v motivating experience and the new website will grow to be much bigger and more community. like the colquhoun thing i think more good will come of it than anything.

    the best thing to come out of it has been some very good offers of help from some v sharp web people, and i really hope they stick around, theres some great stuff we could sort out, will post properly soon (but can barely get in to admin pages and dont want to clog blog with work-in-progress stuff).

    i’ve also got a lot of great new content stuff coming, more to follow. also spending lots of time atm giving talks in various places and spreading the word on putting content free online. i think the royal statistical society for example are hopefully going to put some mp3 stuff up, which is great, the stuff ive been to there has been great. specialists talking to high end generalists is the future.

    thanks again and no more money, to be fair, unless you just want to. i need your time, energy, and banter more than anything. viva etc. a new fast blog, a new bigger site, hurrrah!

  85. Ben Goldacre said,

    July 5, 2007 at 12:21 am

    and get this: i didnt have to set up a vitamin pill company to fund the site. the goodwill economy works. how good is that? i mean really. the world is a good place.

  86. manigen said,

    July 5, 2007 at 9:19 am

    Fantastic.

  87. seenoevil said,

    July 5, 2007 at 9:21 am

    Brilliant. Hope it starts soon.

  88. tom1 said,

    July 5, 2007 at 9:41 am

    Feel free to put my donation towards beer, or a stethoscope made of gold.

    Glad you’ve got it covered. You know where we are if you need us.

    :-)

  89. Frank Chalk said,

    July 5, 2007 at 9:59 am

    Dear Ben,

    As I have made a small fortune selling my patented WIFI Shields to schools who wish to protect their teachers and pupils against homeopathic radiation, I feel duty bound to contribute in order to help you out.

    ps If you want to know more about the role of Crystal Healing, perpetual motion machines and Ley Lines in the new Science curriculum then pop over to http://frankchalk.blogspot.com

    Regards
    Frank Chalk

  90. jburton said,

    July 5, 2007 at 10:19 am

    Dear Ben,

    Good to see you at the RSS yesterday evening accepting your “Excellence in Journalism” award – nice speech and considerate to leave it short so we could get to the drinks!

  91. stever said,

    July 5, 2007 at 10:27 am

    thanks to everyone for helping out.
    And for giving me a nice little holiday from moderating duties.

  92. pun1sh3r said,

    July 5, 2007 at 1:02 pm

    A couple of weeks ago I looked into buying a bad science T-Shirt so I could display my love of science and its badness to the world. However, the T-shirt designs are less than great. If you build it, they will come: Create a T-Shirt design competition – make better T-Shirts and you will sell more of them.

  93. tom1 said,

    July 5, 2007 at 2:09 pm

    For the t-shirt:

    A picture of Ben saying:

    “I ran a double blind, peer reviewed study showing your quack medicine is junk, and all I got is this lousy t-shirt!”

    Tom

  94. coracle said,

    July 5, 2007 at 6:22 pm

    I’ve got a design in the making featuring a very stylish double helix. I’ll have to get my arse into gear and finish it off.

  95. drunkenoaf said,

    July 5, 2007 at 7:23 pm

    You might want to e-mail the b3ta team and ask them about their threaded discussion code– you seem to know the site, so just contact them through their contact us form on the front page.

  96. Gimpy said,

    July 5, 2007 at 8:51 pm

    For the t-shirt:

    I’ve got a picture of Patrick Holford performing an act of onanism on a courgette cock.

  97. tom1 said,

    July 5, 2007 at 10:12 pm

    Ummm… What internet sites have you been visiting Gimpy?

  98. mrstrellis said,

    July 5, 2007 at 11:24 pm

    Aha, Positive Internet saves the day. I sent a desperate text message to them begging for help, so it’s all down to me :-)

    But seriously, I *need* this site to survive. It’s such a valuable resource when my mother in law phones me, again, with an article in The Times, again, saying tomatoes cure cancer, again.

  99. bootboy said,

    July 5, 2007 at 11:36 pm

    Although it seems that Ben is well served already (yay) I can help out on the technical side if needed. I have designed and developed a couple of content management systems over the years, managed pretty large-scale internet infrastructures and know the insides of php, apache and mysql pretty well.

    With your set up the rate limiting factor is definitely going to be the database – fancy caching mechanisms can go so far, but when you reach a certain number of visitors, and as your archive grows (meaning people arrive at diverse pages more frequently – pages which will not be cached), you soon reach a stage where you start saturating connections to shared database hosts.

    The problem with this situation is that the upgrade path is not always straightforward. The reason that you are getting cut off is because you are sucking a disproportionate amount of stuff from the ISP’s big fast database server. You may find that, given the level of traffic, an upgrade to a virtual server makes the site impossibly slow. The only guaranteed solution that leaves you headroom for more growth is a multi-server set up with a dedicated database server. Problem being that that’s expensive.

  100. gadgeezer said,

    July 6, 2007 at 1:48 am

    I feel that the tap-dancing Patrick Holford would be far more family-friendly and less likely to get you barred from theme-parks.

    http://www.holfordwatch.info/2007/07/patrick-holford-and-his-tap-dancing-on.html

    Much as I would enjoy a T-shirt of the international bowel-whisperer. Or a T-shirt of him in a headlock delivered by Sarah Jarvis or Emer Keeling. The man has an uncanny knack for bringing out the rhetoric of annoyance in lively women.

  101. NineTailedFox said,

    July 6, 2007 at 9:12 am

    Sent more money anyway. You deserve it.

  102. Andrew Clegg said,

    July 6, 2007 at 9:49 am

    I remember the courgette one from the forums — nearly fell off my chair. Would definitely buy a t-shirt with that on.

    Andrew.

  103. tom1 said,

    July 6, 2007 at 10:14 am

    Bootboy,

    quite a few of us have been trying to assist Ben, but you’re the first poster who seems to be a specialist on the main issue.

    As I understand it, Ben is off sorting out some alternative hosting with his new best mates at Positive Internet. Presumably with a few more horse power the site should at least be bought some time.

    If Positive Internet are providing enough support to address the longer term issues then cool. If not, then perhaps some community assistance (hopefully intellectual rather than tin) may be necessary if Ben isn’t going to have to spend all his crack money on his website.

    Tom

  104. bootboy said,

    July 6, 2007 at 3:35 pm

    “If Positive Internet are providing enough support to address the longer term issues then cool. If not, then perhaps some community assistance (hopefully intellectual rather than tin) may be necessary if Ben isn’t going to have to spend all his crack money on his website.”

    Agreed. I’ll keep an eye on the site and see if it goes well. Hopefully the web-host will take care of business in the medium term anyway.

  105. pauls said,

    July 6, 2007 at 11:41 pm

    Speaking as a member of the Positive Internet support team, we’ve just got the new server set up, and the details off to Ben this evening. I’m sure he’ll be contacting people for help over the weekend, but we’ll be here to assist too, in the short, medium and (hopefully) long term :-) .

  106. manigen said,

    July 7, 2007 at 10:26 am

    Now that’s a good sign. The support team actually visiting the site, I mean.

  107. tom1 said,

    July 7, 2007 at 12:22 pm

    Wow! “Positive Internet” – does what it says on the tin.

    I’m around and available if anything needs helping with. It sounds pretty well covered though.

  108. nmailer said,

    July 11, 2007 at 3:27 pm

    I’m one of the Directors of Positive. Things seem to be progressing nicely, and the good Doctor has transferred to his new dedicated server. One of our head techies is giving him a call just to make sure everything’s running ok, that the backups are working and so on.

    Some people above have expressed concerns about future capacity. This shouldn’t be a problem. We’re well used to running popular things on GNU/Linux platforms. For example, we did Ricky Gervais’s podcast, and that got us into the Guinness Book of Records for the number of hits it got. Perhaps this site’ll be next ;-)

    Anyway, in the unlikely event that the site does begin to overburden the dedicated machine we’ve given Ben (he was previously in a shared environment), then we’ll give him an even more powerful one, or a cluster. This site is an important candle against the return to medieval darkness, so we won’t be the ones to let it go out.

You must be logged in to post a comment.