I've been MIA for awhile so I'm not sure what's up. Can you tell me the 'adsense crap' was about?
Beyond that, is there anything I can do to help out?
Still feeling somewhat down over the google adsense crap but this is what I'm going to do now..
I've just re-installed a machine at home with centOs 4.4 and have setup a development on it. I'm going to rewrite the stats from scratch. Front end will hopefully end up using AJAX as that's what I'd like to learn..
a very quick example can be seen here
Obviously it's not pretty, I just wrote it in about 10 mins to test the capabilities.
This machine only has 2Gb Ram, so I need to up that at some point to fully exploit what I want to do but at least now it has mysql 4.1 so I can utilize some of those features.
Don't know how long this will take or even if it will ever see the light of day. I may still get tired of it all and just let it all fade away.
There are quite a few things I want to add in, but if anyboy has any input, now is the time to speak up. I may not reply, but I will take all suggestions on-board.
Bok
I've been MIA for awhile so I'm not sure what's up. Can you tell me the 'adsense crap' was about?
Beyond that, is there anything I can do to help out?
"So utterly at variance is destiny with all the little plans of men." - H.G. Wells
My2 Cents
Bok you and Free-DC serve the entire DC community by putting out
The Most Comprehensive and complete Stats
For Every team and Every member for Every project
If you worked for me I am sure I would have to pay tens of thousands of Dollars to get this.
I only feel it fair that we the members that use your Stats you worked so hard to build and maintain
By Supporting you in your efforts to improve them
By at least paying for the cost of all the services we all use here Every Day
I will Think long and hard on ways that we could make this happen and will be open for any ideas that members
could provide
I would absolutely be willing to make regular donations if I were in a position to do so, because I've been a member here for so long and utilise the services offered on a daily basis. Unfortunately I'm a full time student working part time (10hrs/wk) and I don't break even each month! What with a car, cat, rent etc. I have to pay more for those things than I am making from my job so I have to sell things just to make up my losses. I'm not elligible for a student loan, but I do get my course fees paid for.
Bok, would it help if you had a couple more people working on the stats with you to lighten the load a bit? You must be busy as hell.
Have you ever thought about the idea of some basic, free of charge stats and some "premium class", full control, feature rich stats that cost like 20$ for a year?
I agree that your stats are without question the best around, especially 'cause you always reply to user requests almost by the hour.
I've donated in the past and I will continue to do so.
If you need help with things like logo design, gfx design and such things, I'd be more than happy to give things a try.
Do you have any estimate on how long that rewrite would take you? I mean, can you use at least some parts of the old code or is that really a 100% rewrite?
EDIT: The only thing I could think of to add to the current stats would be some sort of a pie chart with all one's projects in the personal stats page. BOINCstats offers something like that, but once you get past the 10 projects mark, it becomes all cluttered and some names are cut off. So to implement a big pie chart into the personal stats pages would be a cool thing IMHO.
More people working on the stats would definitely be good, more than any monetary contribution. Be it actual coding, site design, css, graphics.
As I said, I'm basically constrained by CPU and memory to do some of the things I wish, but perhaps that can be addressed. This is the main reason for going back to the drawing board on a seperate machine. The current stats work fairly well, so I'm not going to mess too much with them in the meantime as they server their purpose.
When I say a total rewrite, well, of course I'll reuse quite a bit of code, though it could use tidying and I've started that already on the backend.
Two things I'm initially going to fix.
1. Date rollover.
This has always been integrated into the gathering script which is a bit silly. It checks for a date change and then does a 'rollover' of certain fields. However it means if there are no updates for instance, it appears that you have done work yesterday when this in fact might have been a week ago. Not too difficult to address, just a seperate script which runs after midnight for all projects. Hardest part of it is that I'll have to code a scheduler to replace the simple cron jobs I use right now.
2. Last update.
I'd like to extend this to hold 10/20 updates. Not particularly difficult again, just add more columns and each time roll them over.
Other things to look at
Now that I'm using a newer version of mysql I can take advantage of some of the newer features.
Insert into ... on DUPLICATE KEY update is one (already done). This saves a select count(*) followed by either an insert or update.
update xx select y from can be used
Other ramblings
Seperating the script into components.
Main one to do the parsing and updating of tables. One to do the project ranking, one to do team ranking etc. It's all in one large script right now. The project ranking is the worst for memory usage especially on the very large projects. Seti for instance, as one way or another it has to get all users into memory at once. I do think this can be broken down into chunks though
say you had 10000 users (very small case)
Select name,score from usertable order by score desc limit 1,1000
update each with project rank
Select name,score from usertable order by score desc limit 1001,1000
update each with project rank
etc etc
That should free up the memory each time but would be just as effective..
Front end in next post.
Bok
Front End.
This is where I need help the most.
Does everyone like the layout currently ?
Is it effectively using the screen real estate? Should the links be moved for instance to allow more width to be used?
Would definable columns and colour schemes be useful?
What I'm looking at, with the help of Dyyryath, is to use Zope for development..it'll be a bit of a learning curve for me, but Dyyryath is very knowledgeable in it so hopefully that will help a lot.
Not too much should change content wise, just expanded on. More charts, but dynamic ones. Fix the user comparison screens, fix up the sessions which have been broken forever.
For instance, I'd like to do the following (and some of this involves back end changes too)
1. Combined BOINC stats - there are challenges involved in this, especially if you want to do it 'real time' every time a boinc project changes which I can get into if anyone is interested. I've already coded it some time ago but it's slow (this is why you see the CPID in statstool)
2. Expand the DCR. I hold way more data than is actually displayed which would be usefuly and interesting to show. Also as I don't like the scoring of the DC-Vault or the DCR (no slant on the Vault, I do actually think highly of it regardless). Not sure this can be addressed easily, but perhaps a decay of scores for finished projects could be done , which I quite like the idea of. I think I read that idea on another forum, TA or Ars perhaps
Probably more.
If anyone is interested in helping out, make it know here and what you can do and I'll keep a list.
BlackMountainCow,
I didn't really consider a premium service. I'm not looking to make money as such to be honest. This is Free-DC after all, all services should be available to everyone, regardless of team. What do I get out of it, well learning all the technologies mainly.
Bok
I Love the way you build your front endDoes everyone like the layout currently ?
Like I said the BEST Stats around
The only thing I would Like to see is a comparison screen where you can pick and compare users or teams
lauren,
there is a way you can compare users, it is on your personal stats page.
bok,
I think your stats are awesome, I like the idea of making the stats screen a bit wider by moving the toolbar from the side to the top.
I wish I could help more, but I dont know anything about programming or anything to do with site design...cash is a little low too being that I am a student. However, if you need number input like that for the team dcr stuff, I can do that. Best of luck and thanks for the best stats around!
That compare function is pretty broken though. It does need to be addressed too.
Your input, adding the teams has been one of the biggest contributions I've had - most welcome. Also AMDave and NeoGen contributed a lot for that..
On the programming side, Darkness Productions has helped me a lot in the past.
Bok
I also like the front end. Good to read, colors are easy on one's eyes. Wouldn't change a thing there.
As birdman2584 said, moving the left menu to the top might indeed help some people with smaller resolutions. My screen has 1600x1200, so that's not a prob for me, but I guess 1024x786 could be cluttered at times.
I guess the biggest obstacle is that people don't really know how to help you. I for myself have no clue of mysql/php coding. Again, as birdman2584 said, if "raw" help is needed, you'll have quite a few people willing to help, me included. Or if you need alpha/beta testers, stress testers, and what have you.
I got one question: how large are the current stats actually? I mean, how large are the scripts and codes and how large is the database, MB/GB wise? Just want to get my head straight, not speculating.
On the backend, the main script is about 2000 lines long. It has pluggable functions which do the parsing of stats for each project.
The frontend consists of something like 50 php scripts some of which can be pretty big
Database wise, not really that big since I don't hold data ad infinitum (only 28 days though I might extend that once again).
There is a total of about 20M+ rows altogether I think. Never counted them..
As I've said , it's mainly ideas I'd like, design, though if any coders are interested they are more than welcome.
p.s. I've already done the date rollover script and the lastupdate field is extended to 24 occurences. Seems to work fine...
Bok
Portuguese guy reporting for duty!
Bok, my master exam season is over so I am willing to help you out with:
-money, need your IBAN number or similar to make an account transfer
-track down teams
-translate your stats into Portuguese, Spanish, French, German and Polish?!?!?
Carlos
See this is what I love about Free-DC
Members standing up to say I am here
if you Need any help
Carlos,Originally Posted by em99010pepe
don't worry about money.
Teams, yes, use the addtean function from the teams.php pages for projects, that certainly would help.. Check with Birdman for how he has been doing it perhaps..
Translation is certainly an interesting idea, let me think on how to do it (css perhaps).
I do have a task for you maybe though, seeing you mentioned German....one set of stats I've never managed to do is MoneyBee as the stats are only available if you are logged in to my knowledge, so I can't get them via LWP. I think you have ran this in the past? Could you perhaps contact them and see if they could make available a csv/txt or xml document I could parse ? All it needs at a mimimum is
unique userid
name (might be the same as above)
team
score
any other metric
Bok
Ok. First priority is to talk to the MoneyBee guys. I will do that right away.Carlos,
don't worry about money.
Teams, yes, use the addtean function from the teams.php pages for projects, that certainly would help.. Check with Birdman for how he has been doing it perhaps..
Translation is certainly an interesting idea, let me think on how to do it (css perhaps).
I do have a task for you maybe though, seeing you mentioned German....one set of stats I've never managed to do is MoneyBee as the stats are only available if you are logged in to my knowledge, so I can't get them via LWP. I think you have ran this in the past? Could you perhaps contact them and see if they could make available a csv/txt or xml document I could parse ? All it needs at a mimimum is
unique userid
name (might be the same as above)
team
score
any other metric
Bok
Carlos
EDIT: Email sent. I used this page contact: http://uk.moneybee.net/impressum.asp
Wow Bok, okay I know a little php and some mysql now Im understanding the syntantical symantics of the whole freaking thing I could help with debugging of crap - I even got an "in the wild" apache server running Apache 2, php 5 with MySQL 4.1.12 running on it
God bless an oldish version of "XAMPP" I can fire up FTP so you can upload / download files and I can obviously set up any database so you can test fire code on a small subset of results. I could do some CSS but ask moogie I truely hate the stuff but It's fun, mind you it's even more fun when the HTML breaks for no reason between browsers :-S
Like many have said before your stats ROCK.and No offence to DP / Dyyryath you even out do them I can remember Dyyryath's Stats pages for seti they were good but flattish.
So anything ya need I try and help - I'll find the time - who knows I may even not play "WoW" one night to code (just order the prozac now!)
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..
SQA Engineer at your service
Bok,
I use your stuff every day, and have sent you a few $$ in the past. However, I also happen to be a professional (some might disagree) SQA guy. I'm conversant with every browser out there, and have been testing web applications since (wait...) 1995. That included 4 years at Netscape back in the 90's. Include me in on any testing needed. Happy to assist.
cheers,
-jim
OK, apparently the first order of business around here needs to be subtracting about 30% off of all gopher_yarrowzoo's project totals...
Actually, you probably remember someone else's SETI stuff. I don't remember ever doing SETI stats, other than a quick overnight hack before Free-DC existed while I was still at ArsTechnica.
I didn't really get into stats until I started doing Folding@Home & Genome@Home stuff. I added them for a few other projects, but mostly I was just interested in providing them for those projects that I enjoyed participating in myself. I wasn't nearly as altruistic as our buddy Bok...
"So utterly at variance is destiny with all the little plans of men." - H.G. Wells
Gopher,
well as Dyyryath has now set up Zope for me on my dev server and is teaching me how to use it, we may not use php at all...it's all python stuff and abstraction from the code. The sql interface he showed me yesterday is REALLY neat. Of course you are welcome to try and learn this too. Another great part about zope is that it can be used as a collaboration tool, multiple people can log in and make changes and such. We've got it open on the net right now, tunnelled through my firewall.
I don't know python personally, but a lot can be achieved without knowing it.. anyway what's another language. Once you know a few it's just a case of learning syntax right...
Oh, that's doable!!OK, apparently the first order of business around here needs to be subtracting about 30% off of all gopher_yarrowzoo's project totals...
Caferace, I'll bear that in mind
Bok
Dyyryath - hmm "http://stats.zerothelement.com/cgi-bin/distributed-folding/render-users.pl?team=5"
^ old link doubt it works but im sure you will remember those stats.
30% off - fine! I will get you for that one shame your not participating in any project Im crunching which at current is SoB just for the hell of it.
Zope sounds like fun I'll go find out about it and maybe play with it a little on my system here.
If you need any CSS help I'll can do that but It can be really annoying if you design it and the freaking thing don't work right or it does in 1 browser and breaks in another for no reason, just ask Dearest Moogie about it.
I was thinking of a layout in my head all "floated" boxes so that everything is in it's place but looks the same in all browsers basically "no tables" all "div" elements - includes for the win me thinks.
Semi-retired from Free-DC...
I have some time to help.....
I need a new laptop,but who needs a laptop when you have a phone...
Now to remember my old computer specs..
Bok I see your adsense is back
Hey Bok,
Here's a crude link to MoneyBee Team Stats: http://us.moneybee.net/mitglieder/st...sp?sp=1&a=6000
Bill
Bok
There are two things I see that would be a Good addition to the stat menu.
#1 A User search input a user name and it take you to the team stat page. You might have to limit the search to projects I don't know
#2 A definable Total The 7 day is good but make the 28 day definable with a pull down menu to do 14 , 21 , 28. 42 days You could keep the 28 day as a default or something like it
I think these ideas would be a good additions to the already Best stats around
The search will be in. In fact Dyyryath was already working on something where it will automatically scroll down to your user in a list of all users and such.... but there will also be a defined search for user of team certainly..
Last X days is also fairly doable as the sql is easily customized..
Bok
I guess Grate minds think alike
So tell us How many lines of code will Dyyryath and you have to write or rewrite to make this all work
Hey Bok. How are you limited by hardware? CPU/Memory? Need some extra grunt? Whats the systems you/are plan on running it on?
memory really.
The current dev box is a dual xeon 3.6 with 2Gb ram.
The prod box is a different matter as it's a dedicated server so getting a dual is costly. But I may just get a business connection and host it at home anyway. Would allow me to use multiple boxes which would be MUCH less constraining
Bok
Bok, what kind of memory do you need? I've got a few 512M sticks from an upgrade to my Xeon box that I'm too lazy to put on eBay:
- qty. 2, DDR266
- qty. 4, DDR400
It's all registered ECC.
Will the new stat pages still have this link
http://www.free-dc.org/utilities/fadprogress/
A little out dated I think
Not really part of the stats though...
Perhaps we'll archive some of these utilities at some point. That was some awesome tool!
Now where did MerePeer go to ..
Bok
Posting Permissions
