I'm going to start the last part of the process of moving the stats and forum over to their new homes today/tomorrow. Hopefully it will be fairly smooth and not much noticeable difference but it's possible things might get missed so let me know.
I'll have to switch off the dnet proxy temporarily whilst I move it's contents. Correspondingly, ppstats will switch off and will be the last thing to switch back on.
The stats have been running parallel on the new server for a few months now. It's quite a bit faster given it's a core i7 and the database tables themselves are housed on solid state drives. The combined BOINC stats are therefore calculated in real time rather than on a slight delay as they currently are. What is missing is the weekly/monthly data. I'll be copying that over in the coming days given it's an 80Gb transfer..
The bandwidth up is lower but we'll just have to see if it handles the load ok, I can't tell right now. If it doesn't I'll look into getting that increased on a business line which will also allow for a static IP, but the costs are fairly high there so I'm trying not to do so unless really necessary..
I'll be switching DNS on the stats site prior to the forum which will mean it will not think you are 'logged in' on the stats until the forum is switched too. Therefore statstool should not really be used until completed...
Bok
It's something I'm going to have to investigate. Likely a 'clear the cookie' issue. Will sort it out tomorrow though. Spent way too much time on all of this today..
Should be fixed now...
Now only thing I have a problem with is :
my firewall does not support putting through custom ports to NAT, so I can't forward port 2064 for the dnetproxy.. it's running on port 23 for now though as it always has in addition.
Connection is not as good as it used to be. IB was paying quite a lot for the old one and I haven't got that kind of spare change to throw around right now. $100-150+ per month extra likely..
Just updated the firmware on my firewall to see if it makes a difference but it hasn't. It's just the amount of traffic coming in when I only have 768K up (15Mbps down). I'd ideally need to upgrade to a business connection and get the 5Mbps up.
Hmm I suppose it would depends on what uses the most bandwidth I'm suspecting the stats pity you couldn't have put that somewhere else with a better connection and made sure you could get to it if need be.
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..
The forum takes little to no resources at all really. Stats uses up 99% of the bandwidth and requires the most use of the hardware too....
Well all we have to do is fine a few more people like me
That are willing to put out some money every month like I do
It is Like DC-ing DC-ers share there resources to help the World
Well giving your resources to Free-DC so they can help the world does the same
I Pledge $50.00 a month I have set it up with my bank Auto-pay to pay once a month
So come on Gay/Gals Look into your Harts and your pockets
And make a pledge to help the Free-DC world
If you have the Bucks to DC You have the Buck to HELP
Thanks Lauren, I do appreciate that.
I've been spending time trying to figure out where the problems are. Found and blocked one IP address which seems to have been trying to get the same file every second or so in bursts, was taking approximately 1/3 of available bandwidth usage. Next I may replace my standard cable modem and also the router I have in place, I just wish I had a good way to check where the problem is occuring. Running ntop on the webserver shows that it can handle the traffic with ease.
Where is that PayPal link here? I saw it when I was not logged in, now I cannot see it. I don't have that much, but I guess that it is 'BTN' (Better Then Nothing).
Sorry — your last action could not be completed
If you were making a purchase or sending money, we recommend that you check both your PayPal account and your email for a transaction confirmation after 30 minutes.
If you came to this page from another website, please return to that site (don't use your browser's Back button) and restart your activity.
If you came from PayPal's website, click the PayPal logo in the upper-left corner to return to our home page and restart your activity. You might have to log in again.
Hmmmm...
Not sure why the paypal link does not work. A few others have used it recently ok??
Thanks Al, got that.
Let me give you all an update
Anyways I'm almost stuck as to what the problems are right now. I do see a lot of traffic so it must be mostly working but there is obviously some bottleneck. I wish I knew someone with more networking experience than I have to help out so if anyone knows please get in touch.
I've just replaced my switch with a $100 netgear gigabit version and it's not made any difference at all that I can tell.
This is the setup right now.
I'm on a 15MBps down/768K up connection. Comes into to a splitter using RG6 which splits the signal into 4 components. One goes to my Motorola SB5100 cable modem, one for voip and the other 2 for cable tv.
From the cable modem it goes into the Netgear FVG318 firewall, from there to the gigabit switch and then to the servers.
From either side of the switch if I ping say 8.8.8.8 I can get anything from 23ms to 1500ms.
Three places I could attempt to alter.
1. Firewall - it should be capable of handling the bandwidth.
2. Cable Modem - I could buy a newer one capable of docsis 3 (mine is docsis 2 )
3. I could try to alter the initial splitter to split it into 2, once going to the cable modem and the second one then splitting out for cable tv and voip.
I mention 3 as I managed to log into the cable modem and the downstream power level seemed a little high
Frequency 609000000 Hz Locked Signal to Noise Ratio 35 dB Power Level -10 dBmV
The Downstream Power Level reading is a snapshot taken at the time this page was requested. Please Reload/Refresh this Page for a new reading
Upstream Value Channel ID 7 Frequency 33000000 Hz Ranged Power Level 42 dBmV
I have a nagging doubt in my mind that DNS requests are somehow causing the problems though. I've been messing with nameservers locally for the last few days and even if I have one on the same machine I'm 50% of the time getting timeouts.
I'm really not sure where to go next and would really appreciate any help people can think of. I did try getting in touch with Dyyryath (old free-dc guru) but he's not responding..
Bok
I've built a smoothwall firewall from an old P4 server I had, been trying to get it configured which is why the site was down but it looks like one of the nics is bad. I'm going to have to buy a new nic for it as I don't seem to have any others in my 'stash'.
I'll probably have to sever the connection again sporadically whilst I get it up and running. This will replace the Netgear Firewall I have which at the moment is at the top of my list for the case of the bandwidth saturation.
Mucho snippage, but ...
Taken me all day to get to post this. Site is V slow. Actually F slow Mostly timeouts today So this is very terse as I've got a feeling I won't get to post it ...
A few random thoughts:
Only 768k upstream? Seems slow. My home use bargain basement cable connection is 1m for only 10m downstream.
From my days working with the UK cable supplier, a four way splitter was a no-no. Three ways max. And that need much tweaking in the street cab to get the levels right.
SB5100 - Isn't that kinda outdated now? I don't think we use them over here any more. Hasn't your ISP given you something better?
Pings to 8.8.8.8 taking 1.5s seems odd. Hardware problem?
Power level - Not familiar with your network levels (only ever done satellite for US clients). 4-way split is going to be 14dB down. Seems odd to have too high level.
DNS - I don't think that's the problem. Download my 'big' graphic is slower than dialup. It's a bandwidth problem.
My guess is failing hardware. But then if your upstream is overloaded I think of malware, but you're not likely to be daft enough to not notice that!
Al.
Appreciate the comments!
I think the UK are ahead of the US in cable connections. I can't get better than 768K up without getting what's termed a business connection and those are not cheap..
7 Mbps down, 786 Kbps up - $97.95/mo.
10 Mbps down, 1 Mbps up - $167.95/mo.
12 Mbps down, 1.5 Mbps up - $279.95/mo.
15 Mbps down, 2 Mbps up - $389.95/mo.
I've got the two way splitter so I can try that part tomorrow I think.
I've had that SB5100 for quite some time so they may be giving out better ones these days. It's the next thing to try anyway even if I buy my own.
If I switch off the http redirect on the firewall pings are a lot better but still I don't think the connection is saturated as such. Get to that in a moment.
I've got the smoothwall server running now in place of the netgear firewall so we'll see if that makes a difference. I'm not going to tweak anything again till tomorrow. Let me know if it appears better or not. It does have much better diagnostics and is currently showing a snapshot of
malware - it's possible but pretty unlikely - I do keep on top of that kind of thing, but I'll certainly check it out. I did see a few kb going to eDonkey when I installed ntop on the webserver but I think that was a red herring. It's a linux box and I never use p2p clients.
I did tweak apache a little as I was very concerned about reverse dns lookups. Found out that the standard logging format includes %h which can cause hostname lookups regardless of the settings and switched that to %a but I don't think it made a difference.
192.1 Kbit/s / 296.4 Kbit/s (Out / In) which is not too bad.
Just been watching the logs and someone has been trying to do some sql injection attacks from China.. Sneaky but it won't work. Banned the ip - I'm going to get more aggressive on doing this kind of thing.
So : next steps if needed.
1. redo rg6 splitter
2. new cable modem
3. give up unless I win the lottery tonight
There is a very real possibility that I'm just not going to be able to host the stats any longer but we aren't there yet. Forum and the other websites I run are not an issue.
Bok
Is it any better? Certainly seems a lot better from within my network. Smoothwall is running quite nicely. Still got some challenges but I'd like feedback first..
Seemed quick enough from here when it loaded a bit faster than that last time I looked, that's for sure. I'll work on the Paypal link on the front page / stat server see what's up with it if you like.
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..
A lot quicker today. Have been able to get to this thread easily each time I tried throughout the day. My big project stats graphic is still pretty slow to download, but quicker than yesterday.
Overall, things are much better
Al.
I think it's going to be ok then. I am seeing a lot of bandwidth activity but it's still not maxing out. So, perhaps now it's time to stat some bandwidth saving issues instead. I think I've spent a little under $1000 in the recent weeks on hardware, that part can take a rest for now, donations have definitely helped
Taking a look at the w3compiler though, free trial, we'll see what it does..
So, is everything still looking good ? No issues with speed at all?
As of about 30mins ago I've finally figured out why my cacheControl wasn't working so all of the javascript files and static images should now be cached locally. This will hopefully speed up the pages quite a bit more I think.. Try it and see.
I'm also considering making the default on the pages to NOT include the graphs but instead have a button to show them if required. This will just be a parameter in the url, so easily accessible still. The graphs take up about 80% of the size of some of the pages so it will drop the footprint of most pages to only a few kb.
Any thoughts?
Graphs off to the side is OK with me
Some of the graphs/charts were hard for me to read
But That maybe just Me
Speed is vary GOOD now
_____Thank You_____
>>>Bok<<<
Yes good plan I guess I should maybe go warm up some nice swishy code to do a nice reveal without having to reload the page
Additional also making all the tags STATIC automagically would that not help with bandwidth so it's not on having to do a call for each tag as they are cached too?
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..
No, making them static would not change the bandwidth as the end result is still the same size. It would save on db calls and such, but that's not an issue. They are cached locally on the server anyway.
For the 'reveal' function you are thinking about, what we have to ensure is that the data is not built at all until the reveal is done.
Say the fullpage with graphs and all is 20k of html + 100k of graphs, we want initially to only send that 20k. If the user hits reveal then the extra data for the graphs is transmitted. I guess what I'm geting at is not to just hide it via javascript. We are aiming to get the pages to load much faster from the client side.
this is actually a real nice resource for testing webpages with lots of ideas...
I'm actually not worried about bandwidth as such as long as it's not affecting speed, really I'm just trying to make the client experience that much better. With the caching of the javascript and css files and saving of graphs when not needed the page load time should be a few seconds at most.
Nope was never thinking of having the graphs hidden but called upon when asked would need to grab the variables for them from someplace but that is doable
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..
Ok Who changed my 3 crowns in the stats from Gold to Pink and why
I have a hard enough time with the way my Mother spelled my name
With 2 Pink crowns I am sure people will think I am a Gal
Where is it showing as pink? They should not have changed at all..
hahaha I'm sorry they are gold Except when you High lite that row
And I had it highlighted ha ha ha
http://stats.free-dc.org/stats.php?page=users&proj=mil
OK since the Move on the main stat page http://stats.free-dc.org/stats.php?page=index
we have 2 Eon's listed
#1 http://stats6.free-dc.org/stats.php?page=proj&proj=eon
#2 http://stats6.free-dc.org/stats.php?page=proj&proj=eon2
And Both are wrong. The right link is
http://stats.free-dc.org/stats.php?page=proj&proj=eon2
We Need to take the ( 6 ) out of the link to make it right
And The old Eon is dead and should be laid to rest
I agree with Lauren on this one. Graeme and his crew now have their own forum at U. Texas.
We no longer need to preserve the EON forum space. Guess we could put up a linky, but even that seems excessive. Anyone wants to find him should be able to. And the stats for EON[0] are truly only historical. It wouldn't break my heart to reclaim that space. Just my .02 worth.
HOME: A physical construct for keeping rain off your computers.
It doesn't link to stats6 for me? All of the stats pages have dynamic links to whichever versions they are on, basically using a variable of $root_page.
Can you check again at http://<a href="http://stats.free-dc...page=index</a>
for the two Eon projects and see where they link to?
We can put the old Eon into Archive then - Done.
I'll switch off the scripts which do the updates tomorrow. Sad day really, I remember writing those scripts whilst in a hotel room in Sao Paulo, Brazil...
OK I found where the problem lies
It is with the BOINC link to Free-DC main stat
that they list in all projects Statistics and leaderboards
http://milkyway.cs.rpi.edu/milkyway/stats.php
There link brings you here http://stats6.free-dc.org/stats.php?page=index
But the right link should be http://stats.free-dc.org/stats.php?page=index
Bok you need to contact Boinc and have them change there link
Or redirect all your file links
If you do not see what I am saying go to any Boinc project and click on there
Statistics link Click on the link to Free-DC stats
and you will see it leads you to a OLD stat page
http://stats6.free-dc.org/stats.php?page=index
ah, no that's me. It links to http://stats.free-dc.org/ which then redirects to stats6. It's an apache vhost config issue. I'll fix.
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..