Stats are down

[Bok]

I'm remote till tomorrow evening.

Two drives are not responding now, which is too much of a coincidence to be a drive problem.

I will see if I can debug it some more tonight.

This is incredible that it only seems to happen when I'm away as I don't travel often!!

[ChertseyAl]

I thought you'd taught the cat how to cycle the power :)

[Bok]

I can cycle the power remotely ok (already have...) but that's not it.

[Dirk Broer]

Site is back up, but there is no updating being done over the day.

Just jumped 300k, but let's see whether this is permanent. I already dropped back (to 337,711,998) before this day at least trice.

[Bok]

I haven't noticed anything wrong today? Seem to be updating ok.

I remote rebooted the box last night and fsck'd all the filesystems and things seemed to be back to normal which is when I switched the stats back on. (almost worries me more that there was an unexplained failure).

are you still seeing issues?

*EDIT* hmm, I think maybe one of the databases might still be having problems. It flip-flops between the two which is why you see it ok then not ok.

*EDIT2* I see a corrupted table on the combined score in that database. It's repairing now.

[gopher_yarrowzoo]

Hmm how much would this cost to cloud host some of the stuff.. or build a in memory buffer so they disc don't get thrashed as much...

[Bok]

WAY WAY WAY too much.

I'd love to put the database in memory to be honest, every now and then I look into servers with 256Gb or even 512GbRam. Just cost way too much! Something like this is close with 144Gb Ram - http://www.ebay.com/itm/DELL-C6100-CLOUD-SERVER-6x-1-8GHz-AMD-6-CORE-HEX-CORE-144GB-RAM-4x-1TB-/181308311505?pt=COMP_EN_Servers&hash=item2a36d147d1 just the processor speed is a bit slow.

But I don;t think the disks failed at all this time as they are perfectly fine and it's too much of a coincidence for them both to fail at the same time. Ever since I changed the way the stats work by round-robbin the databases things have been fine.

I am going to put 2 extra SSD's into the server this week though so they are on real standby. I have them sitting close by in a standby server which I was given - older I7-920 but it's not very stable itself.

[gopher_yarrowzoo]

I am going to put 2 extra SSD's into the server this week though so they are on real standby. I have them sitting close by in a standby server which I was given - older I7-920 but it's not very stable itself.

Not stable - I think it probably just needs some fresh coffee coolent ;)
Well get it stable ;) if you can :D