Majestic12 stats

[Bok]

Added these yesterday, haven't done the signature pieces yet and haven't decided on DCR inclusion just yet.

http://stats.free-dc.org/new/projpage.php?proj=maj

Bok

[PY 222]

So, I've seen only 3 members for Free-DC doing this project.

Are we going all out on this or what?

Seems to me like a previous DC crawler called grub.org.

[Bok]

I believe it's very bandwidth intensive unlike Dimes, though I'm yet to install it myself yet. I'll try it later on today. Might not be a good idea to put it on many machines on one network if this is the case.

Bok

[PY 222]

I'll give it a try too to see if this will play well with FaD.

Let the games begin.

[PCZ]

I ran this for a couple of days.
It will eat all your bandwidth.

Just dont go there OK

[PY 222]

Originally posted by PCZ
I ran this for a couple of days.
It will eat all your bandwidth.

Just dont go there OK

I read that you can throttle the bandwidth usage on the client.

I'll give it a go and report back soon.

[magnav0x]

Yes you can throttle the bandwith on the client. It does use a lot of bandwidth if you don't. Otherwise it runs nice next to other DC projects. Been running in cunjunction with Prime Sierpinski Project Sieve and PRP, but I'm unsure how long I'll stick with majestic. I prefer more traditional DC projects. I typically only have Majestic open a few hours a day, because I close it a lot

[PY 222]

My node is holding steady at around 600Kbit/s down and it seems to play well with FaD.

The only time its taking up alot of processing power is during the Archiving stage when it is ready to upload data to the server.

I've only ran this for a while, like 2 hours, so I will report back tomorrow.

magnav0x, how many workers do you have and what kind of connection do you set on your node?

[magnav0x]

Originally posted by magnav0x
Yes you can throttle the bandwith on the client. It does use a lot of bandwidth if you don't. Otherwise it runs nice next to other DC projects. Been running in cunjunction with Prime Sierpinski Project Sieve and PRP, but I'm unsure how long I'll stick with majestic. I prefer more traditional DC projects. I typically only have Majestic open a few hours a day, because I close it a lot

I have one that was on all the time with default bandwith settings. And one on my work laptop, that a rarely let run, also with default settings. I'll probably uninstall it from my laptop and let it run on that one machine. Doubt I'll put it on any more machines.

[sTs950]

So far i have completed 19 buckets... only problems is i still havn't completed one bucket upload back to the main server. Anyone have problems like this? i'm ~50% complete 1 bucket (4mb) in 2 hours... I have 1mb outbound bandwidth so i dont think the limit is on this side... played with upload chunk sizes but nothing seems to help... any ideas?

Thanks,
Ryan

[sTs950]

huh.. must be a bad route from comcast to their servers... installed the client on a machine at the office and i get quite nice upload speeds...

Thanks,
Ryan

[PY 222]

Originally posted by sTs950
huh.. must be a bad route from comcast to their servers... installed the client on a machine at the office and i get quite nice upload speeds...

Thanks,
Ryan

Wow... your stats are awesome sTs950 and you've got only two machines on it.

How many workers do you have and what kind of connection do you have?

[sTs950]

ahh, why thank you py =) got a system at work running on a 8mb/1mb cable line running 200 workers(routing my upload to mystic results server threw a backup 1.5mbit dsl line... for some reason the route into the mystic results server (another cable modem) threw comcast is terable.. no problems threw the dsl though). Also have a 10/10mbit connection at a client location that i am also running 200 workers on. They are still not using anywhere need full line capacity though.. prolly averaging about 30 - 60%.. dunno if i need more workers or if i am hitting some other sort of limit..

Thanks,
Ryan

::EDIT:: Also running 10 buckets up from the 5 default. that also seems to help the score a little..

[PY 222]

No wonder I am not getting the scores that you are getting. I have only 25 workers on mine and I already thought it was sufficient.

I'll definitely bump up the worker and bucket to increase the usage of my nodes.

Thanks for the tip sTs950.

[sTs950]

client acts wierd to... ever though i have it set to 200 workers and 10 buckets right now on the one machine it only has 4 buckets open and is running ~50 - 60 workers... I am also running external rar compression at a level of 9... not sure if this helps or not... when the forum posts were created on the subject it looks like they were running a diffrent copression sceme then the LZMA that is built in now (plus there are rar licencing issues..)

Thanks,
Ryan

[PCZ]

Since you all seem to be set on eating up your internet bandwidth' i'll give you a tip
your bottleneck will be the ability to return buckets.
Make sure you are using rar to archive the buckets.

This of course is processor intensive so the CPU needs to be fast or it will become the bottleneck.

If you are using a corporate pipe then think seriously about how much you need your job.
If you can live without that monthly pay check then carry right on if not then stop now.

[sTs950]

you lose you job over this project pcz? I see that at one time you had quite the output on this project. Also, I am the admin on the client network... if they see a proble they come to see me... plus it's the weekend (wont be running this during the work day!) so i can afford to give the router a little work out.. =) just got a new cisco 3845 cause the 2621 couldn't handel the 10/10 =)

Thanks,
Ryan

:::EDIT::: you talking about the kind of content this thing is looking at PCZ? or just the traffic usage? If it's content it's gone in a sec... cant have that kinda stuff... heh

[PCZ]

I am talking about the bandwidth used.

At home it's up to you to use up all the bandwidth or not.
You might run into monthly limits or get your account suspended for abuse though.

At work it's a different story.
Bandwidth costs and is accounted for.
MRTG graphs will show up increased utilisation and someone will ask why.

Also search engines such as Majestic12 are not popular with a lot of Server admins and abuse messages will soon start arriving,

As i said i only run this for a few days.
It was easy to get high up in the dailies because i do have shed loads of bandwidth but if sustained it would have been spotted and i value my job above the kudos of being number 1 in a project of questionable merit.

[maefly]

Yeah, this client is certainly a bandwidth hog...if you let it be.

It took me a couple days of tweaking it, but I got a good compromise between performance and not allowing it to dominate my Internet connection.

I have a 10mb/768kb connection at home and I have it configured as follows:
75 workers
5 buckets
Limit downstream to 3mb
No limit on upstream
Rar compression

I don't know if it's my router or what, but if I allow the downstream to be a bit higher, my upstream is constantly saturated. I don't think the upstream limitations take into account the upload bandwidth of the workers just going out to make a connection with the websites.

So, it takes a little babysitting at first, but once tweaked out, it's easy to leave alone and not notice it. The author of the client really has thought about the end-user and made sure to give you a ton of options to control the way the client handles things.

I would never put this on a work connection. I would saturate that T1 connection in no time!

Jeff

[PCZ]

Try it on a server with teamed gig nics and multiple CPU's to do the rars.
Oh and multiple OC 48's to make sure the inet connection isn't the bottlemeck.

Very career limiting

[CaptainMooseInc]

I've restarted this project. I couldn't get the older client to work a few months back but now it seems to work just fine. Got it up and running and crunching. The only problem is I dunno how to join the Free-DC team. Under Website I put: www.free-dc.org in hopes that it will join the team that way.

If it's done another lemme know. Hopefully I'll show up in the stats soon.

-Jeff

[maefly]

To join a team, follow these steps:
1. Go to http://majestic12.kicks-ass.org/login.jhh and login
2. Click on "Teams"
3. In the "Action" column click on the "Join" link for the team that you want to join

[CaptainMooseInc]

added myself to the team. i currently have 5 buckets open. the program is saying 4 nearly completed a 1 dense. does it have to complete all 5 buckets before it uploads because nothing has uploaded yet...

Also I've noticed that when the program starts archiving the barrels that it locks up and I can't do anything with it. I went to work for 3 hours today and came back to find it totally idle doing nothing....

This is so f'in odd.

-Jeff

[edit] I sat through and waited for the stupid thing to archive for a few, this time it managed to not fully lock up. How are stats calculated for this thing? Is it by barrels or by MBs or what? In the few hours it's been running I've gotten 607MBs down in the 5 open barrels I have. If I increase barrel size to like 10 will it make any difference? I'm just trying to find some way to get some initial stats in. Also Bok, when are you adding MJ12 to the sig??? [/edit]

[CaptainMooseInc]

Well after letting the stupid thing run for a day it's managed to eat over 1GB of downstream, but now only shows only 92 for the "month". 5 buckets marked as "nearly done" with 2 in the dense category, nothing is completing or submitting and my max open buckets is set to 5. This is BS and I'm already sick of this project not working like it's supposed to. I tried it once months ago and it didn't work at all. Now this stupid thing is eating my bandwith, not uploading anything, then bandwith it has eaten is disappearing. Plus it's dropping the performance of Proth_sieve from 110kp/s to 75kp/s and it's not supposed to be a CPU intensive program but for a few minutes an hour while it caches.

Ya'll can have MJ12 back, these guys are morons.

-Jeff

[PY 222]

I am sure its just a setting issue on your node. I've been running MJ12 on two machines for two days now ands its running fine.

How many workers do you have and what is your connection speed?

As for using processing power, yes it does when the node archives the barrels to get uploaded.

[CaptainMooseInc]

I've got an 8mbps downstream w/ 1mb upstream connection (Cable).

I selected the 4096 down connection w/ a limit of 30%.

Maximum open buckets is 5. Maximum number of async workers is 15.

[CaptainMooseInc]

found the problem...it's proth_sieve. Both programs are set to idle priority so MJ12 can archieve but it can't get enough CPU Cycles to start uploading. I stopped proth_sieve and now it's uploading nodes. I'm gonna try to find out how many buckets my computer can go through in a day and then up my number of buckets to cover a day's worth of work. Maybe I'll up the priority on MJ12 node to Low or Normal instead of Idle so it can steal CPU Cycles as needed.

-Jeff

[PY 222]

CaptainMooseInc, glad you found the issue is with the priority.

Just saw you uploading 40k URLs.

[PY 222]

Bok, are we going to be adding this project to the stats page?

[PCZ]

PY
You nicked my 1 spot

Stats would be good.
Did someone say DCR ?

[PY 222]

Originally posted by PCZ
PY
You nicked my 1 spot

Stats would be good.
Did someone say DCR ?

You are more than welcome to come grab it back but I warn you, I bite.

DCR would be lovely. Who knows, I might take No. 1 in DCR as well. Now wouldn't that be lovely.

[PCZ]

Thats OK i brought a muzzle

[Bok]

It's on my list...

Bok

[PY 222]

Originally posted by PCZ
Thats OK i brought a muzzle

Uh oh. You're serious about catching me aren't you. I see that you've started ramping up again.

Catch me if you can...

[PY 222]

OMG... Looks like you've brought out everything you've got.

[PCZ]

No that would be excessive

[PY 222]

Originally posted by PCZ
No that would be excessive

Its never too excessive

I've matched you by turning on one of my other machines.

So, your turn to raise.

[PY 222]

Looks like you've raised your stakes at the game.

Ok then.... I'll go find a box some where to re-raise you.

[LAURENU2]

Originally posted by PY 222
Looks like you've raised your stakes at the game.

Ok then.... I'll go find a box some where to re-raise you.

PY You keep distracted in here and I will sneak up on him over in Rosetta@home

[Bok]

Lauren,

this project doesn't really use CPU cycles, just bandwidth So it can run alongside Rosetta...

Bok