The boring life of Jerod Poore, Crazymeds' Chief Citizen Medical Expert.

Crazy Meds back online

I currently don't know why or for how long Crazy Meds was unavailable, but we're back now.  When I do know I'll post something.

EDIT: Fuck if I know what happened.  The server went down at 2:43 a.m. Mountain Time (09:43 GMT) and I brought it back up at 1:45 p.m. (20:45 GMT).  I've been through every freaking log there is and nothing explains why a shutdown was ordered.

4 comments:

Jacqui said...

And that's why I don't host my own blog. You're a better man than I am, Crazy Meds.

eric said...

I noticed that you run crazymeds on a VPS. Most VPS hosting providers that run Virtuozzo (Parallels VZ, your VPS container control panel URL will end in /vzcp, most likely) as their VPS system will have some sort of auto-throttle system.

For instance, at the company for which I work*, if the 15 minute load average for a customer is over 30 (meaning the load has been at least 30 for at least 15 min), the container is automatically stopped for a 10 minute "cool down" period. At the end of that period, the container is automatically restarted. Theoretically, this all happens without a hitch, and 99.9% of the time, that's the case. However, in a rare occasion, there will be a container that glitches and does not restart properly for whatever reason.

That may have been the case, so you may wish to check with your provider. They'll also probably be able to check the master node logs to see if they can get a better idea of what happened.

Also, if you want to auto-track your load avg so that you know what it was at the time your container stopped, try using this script:
#!/bin/bash
while true
do
if [ ! -e /root/loadavg ]
then
touch /root/loadavg
fi
cat /proc/loadavg > /root/load.avg
sleep 1m
done

When your container goes down, the load average listed would be within the past 60 seconds, so you'll know if it was a load issue causing the problem.

Hope this helps.
(*Note, no I do not work for your hosting provider, and yes, our automated "cool down" script also automatically notifies users when they get cooled down.)

Unknown said...

Eric,

I'm aware of the throttling. If I had the money I'd pay for a full-on private server, because Invision can be a resource hog at times.

I couldn't find any evidence of a resource spike.

Anything can happen once, or even twice. If it happens again I'll get tech support involved. If the load average spiked I want to know what and why so it doesn't happen again.

Unknown said...

And, yes, it is Virtuozzo running under Plesk.