PDA

View Full Version : Majestic12 stats



Bok
12-02-2005, 10:53 AM
Added these yesterday, haven't done the signature pieces yet and haven't decided on DCR inclusion just yet.

http://stats.free-dc.org/new/projpage.php?proj=maj

Bok

PY 222
12-02-2005, 12:07 PM
So, I've seen only 3 members for Free-DC doing this project.

Are we going all out on this or what?

Seems to me like a previous DC crawler called grub.org.

Bok
12-02-2005, 12:35 PM
I believe it's very bandwidth intensive unlike Dimes, though I'm yet to install it myself yet. I'll try it later on today. Might not be a good idea to put it on many machines on one network if this is the case.

Bok

PY 222
12-02-2005, 12:59 PM
I'll give it a try too to see if this will play well with FaD.

Let the games begin.

PCZ
12-02-2005, 01:16 PM
I ran this for a couple of days.
It will eat all your bandwidth.

Just dont go there OK

PY 222
12-02-2005, 02:26 PM
Originally posted by PCZ
I ran this for a couple of days.
It will eat all your bandwidth.

Just dont go there OK

I read that you can throttle the bandwidth usage on the client.

I'll give it a go and report back soon.

magnav0x
12-02-2005, 04:05 PM
Yes you can throttle the bandwith on the client. It does use a lot of bandwidth if you don't. Otherwise it runs nice next to other DC projects. Been running in cunjunction with Prime Sierpinski Project Sieve and PRP, but I'm unsure how long I'll stick with majestic. I prefer more traditional DC projects. I typically only have Majestic open a few hours a day, because I close it a lot :rotfl:

PY 222
12-02-2005, 04:16 PM
My node is holding steady at around 600Kbit/s down and it seems to play well with FaD.

The only time its taking up alot of processing power is during the Archiving stage when it is ready to upload data to the server.

I've only ran this for a while, like 2 hours, so I will report back tomorrow.

magnav0x, how many workers do you have and what kind of connection do you set on your node?

magnav0x
12-02-2005, 04:40 PM
Originally posted by magnav0x
Yes you can throttle the bandwith on the client. It does use a lot of bandwidth if you don't. Otherwise it runs nice next to other DC projects. Been running in cunjunction with Prime Sierpinski Project Sieve and PRP, but I'm unsure how long I'll stick with majestic. I prefer more traditional DC projects. I typically only have Majestic open a few hours a day, because I close it a lot :rotfl:

I have one that was on all the time with default bandwith settings. And one on my work laptop, that a rarely let run, also with default settings. I'll probably uninstall it from my laptop and let it run on that one machine. Doubt I'll put it on any more machines.

sTs950
12-02-2005, 06:48 PM
So far i have completed 19 buckets... only problems is i still havn't completed one bucket upload back to the main server. Anyone have problems like this? i'm ~50% complete 1 bucket (4mb) in 2 hours... I have 1mb outbound bandwidth so i dont think the limit is on this side... played with upload chunk sizes but nothing seems to help... any ideas?

Thanks,
Ryan

sTs950
12-02-2005, 09:26 PM
huh.. must be a bad route from comcast to their servers... installed the client on a machine at the office and i get quite nice upload speeds...

Thanks,
Ryan

PY 222
12-03-2005, 01:22 AM
Originally posted by sTs950
huh.. must be a bad route from comcast to their servers... installed the client on a machine at the office and i get quite nice upload speeds...

Thanks,
Ryan

Wow... your stats are awesome sTs950 and you've got only two machines on it.

How many workers do you have and what kind of connection do you have?

sTs950
12-03-2005, 02:55 AM
ahh, why thank you py =) got a system at work running on a 8mb/1mb cable line running 200 workers(routing my upload to mystic results server threw a backup 1.5mbit dsl line... for some reason the route into the mystic results server (another cable modem) threw comcast is terable.. no problems threw the dsl though). Also have a 10/10mbit connection at a client location that i am also running 200 workers on. They are still not using anywhere need full line capacity though.. prolly averaging about 30 - 60%.. dunno if i need more workers or if i am hitting some other sort of limit..

Thanks,
Ryan

::EDIT:: Also running 10 buckets up from the 5 default. that also seems to help the score a little..

PY 222
12-03-2005, 03:12 AM
No wonder I am not getting the scores that you are getting. I have only 25 workers on mine and I already thought it was sufficient.

I'll definitely bump up the worker and bucket to increase the usage of my nodes.

Thanks for the tip sTs950. :thumbs:

sTs950
12-03-2005, 03:24 AM
client acts wierd to... ever though i have it set to 200 workers and 10 buckets right now on the one machine it only has 4 buckets open and is running ~50 - 60 workers... I am also running external rar compression at a level of 9... not sure if this helps or not... when the forum posts were created on the subject it looks like they were running a diffrent copression sceme then the LZMA that is built in now (plus there are rar licencing issues..)

Thanks,
Ryan

PCZ
12-03-2005, 03:35 AM
Since you all seem to be set on eating up your internet bandwidth' i'll give you a tip
your bottleneck will be the ability to return buckets.
Make sure you are using rar to archive the buckets.

This of course is processor intensive so the CPU needs to be fast or it will become the bottleneck.

If you are using a corporate pipe then think seriously about how much you need your job.
If you can live without that monthly pay check then carry right on if not then stop now.

sTs950
12-03-2005, 03:44 AM
you lose you job over this project pcz? I see that at one time you had quite the output on this project. Also, I am the admin on the client network... if they see a proble they come to see me... plus it's the weekend (wont be running this during the work day!) so i can afford to give the router a little work out.. =) just got a new cisco 3845 cause the 2621 couldn't handel the 10/10 =)

Thanks,
Ryan

:::EDIT::: you talking about the kind of content this thing is looking at PCZ? or just the traffic usage? If it's content it's gone in a sec... cant have that kinda stuff... heh

PCZ
12-03-2005, 04:14 AM
I am talking about the bandwidth used.

At home it's up to you to use up all the bandwidth or not.
You might run into monthly limits or get your account suspended for abuse though.

At work it's a different story.
Bandwidth costs and is accounted for.
MRTG graphs will show up increased utilisation and someone will ask why.

Also search engines such as Majestic12 are not popular with a lot of Server admins and abuse messages will soon start arriving,

As i said i only run this for a few days.
It was easy to get high up in the dailies because i do have shed loads of bandwidth but if sustained it would have been spotted and i value my job above the kudos of being number 1 in a project of questionable merit.

maefly
12-03-2005, 11:53 AM
Yeah, this client is certainly a bandwidth hog...if you let it be.

It took me a couple days of tweaking it, but I got a good compromise between performance and not allowing it to dominate my Internet connection.

I have a 10mb/768kb connection at home and I have it configured as follows:
75 workers
5 buckets
Limit downstream to 3mb
No limit on upstream
Rar compression

I don't know if it's my router or what, but if I allow the downstream to be a bit higher, my upstream is constantly saturated. I don't think the upstream limitations take into account the upload bandwidth of the workers just going out to make a connection with the websites.

So, it takes a little babysitting at first, but once tweaked out, it's easy to leave alone and not notice it. The author of the client really has thought about the end-user and made sure to give you a ton of options to control the way the client handles things.

I would never put this on a work connection. I would saturate that T1 connection in no time!

Jeff

PCZ
12-03-2005, 12:23 PM
Try it on a server with teamed gig nics and multiple CPU's to do the rars.
Oh and multiple OC 48's to make sure the inet connection isn't the bottlemeck.

Very career limiting :D

CaptainMooseInc
12-04-2005, 09:35 AM
I've restarted this project. I couldn't get the older client to work a few months back but now it seems to work just fine. Got it up and running and crunching. The only problem is I dunno how to join the Free-DC team. Under Website I put: www.free-dc.org in hopes that it will join the team that way.

If it's done another lemme know. Hopefully I'll show up in the stats soon.

-Jeff

maefly
12-04-2005, 10:31 AM
To join a team, follow these steps:
1. Go to http://majestic12.kicks-ass.org/login.jhh and login
2. Click on "Teams"
3. In the "Action" column click on the "Join" link for the team that you want to join

CaptainMooseInc
12-04-2005, 03:41 PM
added myself to the team. i currently have 5 buckets open. the program is saying 4 nearly completed a 1 dense. does it have to complete all 5 buckets before it uploads because nothing has uploaded yet...

Also I've noticed that when the program starts archiving the barrels that it locks up and I can't do anything with it. I went to work for 3 hours today and came back to find it totally idle doing nothing....

This is so f'in odd.

-Jeff

I sat through and waited for the stupid thing to archive for a few, this time it managed to not fully lock up. How are stats calculated for this thing? Is it by barrels or by MBs or what? In the few hours it's been running I've gotten 607MBs down in the 5 open barrels I have. If I increase barrel size to like 10 will it make any difference? I'm just trying to find some way to get some initial stats in. Also Bok, when are you adding MJ12 to the sig???

CaptainMooseInc
12-05-2005, 08:21 AM
Well after letting the stupid thing run for a day it's managed to eat over 1GB of downstream, but now only shows only 92 for the "month". 5 buckets marked as "nearly done" with 2 in the dense category, nothing is completing or submitting and my max open buckets is set to 5. This is BS and I'm already sick of this project not working like it's supposed to. I tried it once months ago and it didn't work at all. Now this stupid thing is eating my bandwith, not uploading anything, then bandwith it has eaten is disappearing. Plus it's dropping the performance of Proth_sieve from 110kp/s to 75kp/s and it's not supposed to be a CPU intensive program but for a few minutes an hour while it caches.

Ya'll can have MJ12 back, these guys are morons.

-Jeff

PY 222
12-05-2005, 12:34 PM
I am sure its just a setting issue on your node. I've been running MJ12 on two machines for two days now ands its running fine.

How many workers do you have and what is your connection speed?

As for using processing power, yes it does when the node archives the barrels to get uploaded.

CaptainMooseInc
12-05-2005, 01:57 PM
I've got an 8mbps downstream w/ 1mb upstream connection (Cable).

I selected the 4096 down connection w/ a limit of 30%.

Maximum open buckets is 5. Maximum number of async workers is 15.

CaptainMooseInc
12-05-2005, 02:13 PM
found the problem...it's proth_sieve. Both programs are set to idle priority so MJ12 can archieve but it can't get enough CPU Cycles to start uploading. I stopped proth_sieve and now it's uploading nodes. I'm gonna try to find out how many buckets my computer can go through in a day and then up my number of buckets to cover a day's worth of work. Maybe I'll up the priority on MJ12 node to Low or Normal instead of Idle so it can steal CPU Cycles as needed.

-Jeff

PY 222
12-05-2005, 02:34 PM
CaptainMooseInc, glad you found the issue is with the priority.

Just saw you uploading 40k URLs. :|party|:

PY 222
12-06-2005, 01:05 AM
Bok, are we going to be adding this project to the stats page?

PCZ
12-06-2005, 02:35 AM
PY
You nicked my 1 spot :D

Stats would be good.
Did someone say DCR ?

PY 222
12-06-2005, 02:40 AM
Originally posted by PCZ
PY
You nicked my 1 spot :D

Stats would be good.
Did someone say DCR ?

You are more than welcome to come grab it back but I warn you, I bite. :D

DCR would be lovely. Who knows, I might take No. 1 in DCR as well. Now wouldn't that be lovely.

PCZ
12-06-2005, 02:53 AM
Thats OK i brought a muzzle :jester:

Bok
12-06-2005, 07:31 AM
It's on my list...

Bok

PY 222
12-06-2005, 11:39 AM
Originally posted by PCZ
Thats OK i brought a muzzle :jester:

Uh oh. You're serious about catching me aren't you. I see that you've started ramping up again.

Catch me if you can... :jester:

PY 222
12-06-2005, 08:44 PM
OMG... Looks like you've brought out everything you've got.

:notworthy

PCZ
12-07-2005, 11:32 AM
No that would be excessive :jester:

PY 222
12-07-2005, 12:36 PM
Originally posted by PCZ
No that would be excessive :jester:

Its never too excessive ;)

I've matched you by turning on one of my other machines.

So, your turn to raise. :|party|:

PY 222
12-07-2005, 03:30 PM
Looks like you've raised your stakes at the game.

Ok then.... I'll go find a box some where to re-raise you. :jester:

LAURENU2
12-07-2005, 04:56 PM
Originally posted by PY 222
Looks like you've raised your stakes at the game.

Ok then.... I'll go find a box some where to re-raise you. :jester:
:idea: PY You keep distracted in here and I will sneak up on him over in Rosetta@home :cheers: :jester:

Bok
12-07-2005, 05:01 PM
Lauren,

this project doesn't really use CPU cycles, just bandwidth :) So it can run alongside Rosetta...

Bok :cheers:

PY 222
12-07-2005, 07:50 PM
Holy smokes PCZ, you've got 8 nodes doing this now.

Argggghhh and here I was thinking I could pwned on this Project.

Looks like Free-DC is really smoking up the charts with all of us crunching. :thumbs: :thumbs:

PCZ
12-07-2005, 08:05 PM
It's 7 really.

Had a bit of bother with PCZ-01 and moved the MJ12 directory to another box.
Now it shows up twice. :(

Just had a look at one of my nodes which has been running for 24 hrs.
42GB downloaded 1.1GB uploaded. :shocked:

CaptainMooseInc
12-07-2005, 09:21 PM
once again i'm done w/ this project. got it running but i get some weird error about the database being locked out (or up?) everytime it runs for like 10-20 mins alongside proth_sieve. When i leave proth_sieve off it can run forever w/o problem. Obviously the two projects are conflicting (or Comcast is doing something to kill a program like MJ12 to stop bandwith consumption). I dunno but i'm not giving up RieselSieve for MJ12.

-Jeff

PY 222
12-08-2005, 12:51 PM
Hate to see you go CaptainMooseInc but we all understand if you cannot crunch this project. This is Free-DC for a reason. :D

Anyway PCZ, I am starting to choke on your fumes. Trailing you so close in the dailies sure ain't fun. I add one node, you add one more. Argggggh :cuss:

When can I overtake you in dailies?????

Also, welcome Bok back to the party and birdman2584.

This is begining to look like a real Free-DC team. Maybe a new forum for Majestic-12?

Bok
12-08-2005, 01:22 PM
You got it :)

I've shut down my node. Will load it up on a different machine as it was on my windows desktop and was affecting a few other things.. I'll put it on a cruncher instead.

Bok

PY 222
12-08-2005, 01:35 PM
Originally posted by Bok
You got it :)

I've shut down my node. Will load it up on a different machine as it was on my windows desktop and was affecting a few other things.. I'll put it on a cruncher instead.

Bok

Make sure you limit the amount of bandwidth your node can use or else it'll suck it all up. :D

n7vxj
12-09-2005, 01:58 AM
Any suggestions on what I should set my bandwidth usage to? this is my main box that I use for my internet,etc...I'm on a a 256 dsl connection, if that matters.

OOPS!! Sorry Bok, just realized this is posted in the wrong place!! HAs nothing to do with stats!!:bang:

PCZ
12-09-2005, 04:02 AM
PY

I am going to be cutting back a bit on this.
I just did the numbers on my data transfer yesterday and it ain't gonna fly. :mad:

IronBits
12-09-2005, 05:27 AM
Sounds a bit more like a BIG FAT PIPE test :eek:

Longbow
12-09-2005, 06:04 PM
Originally posted by PCZ
I ran this for a couple of days.
It will eat all your bandwidth.

Just dont go there OK

I downloaded this and ran it for about 3 to 4 hours and it consumed over 2 Gig in this short span. This made me start thinking about what exactly is too much in my ISP's eyes. After a bit of searching I found out that the Cox@home service allows 2 Gig per day and/or 40 Gig per month.

Needless to say I won't be doing any more Majestic12. And I would suggest anyone doing so take a good look at their ISP's acceptable use policy.

KWSN_Dagger
12-09-2005, 06:12 PM
That's why i stopped as well.. I'm only allowed 30Gb a month.

CaptainMooseInc
12-12-2005, 12:21 AM
Well i've dug through and found Comcast's acceptable use policy. It says nothing about maximum up/downstreams and basically all it says is don't download illegal stuff.

Majestic-12 isn't illegal as far as I know.

I've knocked the priority for the program to Below Normal and set a downstream limit of 2400kbps on my 8Mbps connection. I'm losing 30-40kp/s on proth_sieve but I feel like dealing with this for a bit and seeing how long it takes Comcast to kill my internet connection and I have to call and b**** them out because they don't have written limitations. Hmmm, maybe I could sue them (the american way) and make millions for cutting me off from something I paid for and they didn't set limits. :-D

I turned it back on around 3PM and it's 12:25AM now....6.6GBs downloaded, 130MBs uploaded :)

Take THAT Comcast. :-D

-Jeff

IronBits
12-12-2005, 12:53 AM
Better read it again :p
http://www.comcast.net/terms/use.jsp

viii. restrict, inhibit, interfere with, or otherwise disrupt or cause a performance degradation, regardless of intent, purpose or knowledge, to the Service or any Comcast (or Comcast supplier) host, server, backbone network, node or service, or otherwise cause a performance degradation to any Comcast (or Comcast supplier) facilities used to deliver the Service

x. connect multiple computers behind the cable modem to set up a LAN (Local Area Network) that in any manner would result in a violation of the terms of this Policy or an applicable Service plan --- this one could get some of us in trouble ;)

xxii. interfere with computer networking or telecommunications service to any user, host or network, including, without limitation, denial of service attacks, flooding of a network, overloading a service, improper seizing and abuse of operator privileges and attempts to "crash" a host

I think the term 'flooding of a network' would include this PHAT pipe testing project. Be careful!

CaptainMooseInc
12-12-2005, 10:43 AM
Well I don't see DEFINED limitations (like Cox's). I'm at nearly 14GBs downstream now since 3PM yesterday!!!

Booyah.

-Jeff

PY 222
12-12-2005, 12:46 PM
I definitely wouldn't recommend anybody that has a bandwidth limitation to run this project because it is a bandwidth hogger.

em99010pepe
12-12-2005, 12:57 PM
I prefer to spend my 4GB international bandwidth in **legal downloads. :D

PY 222
12-12-2005, 01:10 PM
Originally posted by em99010pepe
I prefer to spend my 4GB international bandwidth in **legal downloads. :D

Your bandwidth quota for a month is only 4GB? And what do you mean by international bandwidth? How can the ISP know whether its international or not?

em99010pepe
12-12-2005, 01:21 PM
Originally posted by PY 222
Your bandwidth quota for a month is only 4GB? And what do you mean by international bandwidth? How can the ISP know whether its international or not?

Yes, it is. Don't forget that I live in the third world, PORTUGAL.
The ISP can detect if the traffic is national or international. Now my national bandwidth is unlimited but the international is limited.
Everytime I open Free-DC page I am spending my international bandwidth !!!!
There's an option in the forum to block avatars/signatures and so on. I'm using it right now.

National= portuguese traffic
International= the rest of the world

Carlos

PY 222
12-12-2005, 02:05 PM
And what will happen if you use a proxy to browse international sites?

em99010pepe
12-12-2005, 02:27 PM
Originally posted by PY 222
And what will happen if you use a proxy to browse international sites?

Good question. I don't know.

Carlos

Longbow
12-12-2005, 07:30 PM
Originally posted by CaptainMooseInc
Well I don't see DEFINED limitations (like Cox's).
-Jeff

Check out this article. Seems thier undefined (yet enforced) limits are causing quite a stir.

http://www.post-gazette.com/pg/04033/268244.stm

CaptainMooseInc
12-13-2005, 05:21 AM
well i wanted a definate answer so I called Comcast and they had to call me back in 10 mins when i asked the question...

I told them I was going to probably use 100-200GBs/week worth of downstream. They said that they base their policy on it "affecting other users on the network" and that the estimates I was giving them should not affect other users and therefore I -shouldn't- be disconnected. But he added a disclaimer that an automatic system will determine whether I'm degrading other user's service and shut me off if my demand becomes to great but that I can call them and they'll hook me back up...but very unlikely that any of that would happen.

-Jeff

em99010pepe
12-13-2005, 12:17 PM
Today I went to university and I added one machine to Majestic. My problem now is how to join Free-DC. I can't login using the page maefly gave a few posts behind!!!

Carlos

PY 222
12-13-2005, 01:01 PM
Go to this link: http://majestic12.kicks-ass.org/teams/team.jhh?teamid=16&join=1

Enter your email address and password that you've used for the node.

em99010pepe
12-13-2005, 01:16 PM
Originally posted by PY 222
Go to this link: http://majestic12.kicks-ass.org/teams/team.jhh?teamid=16&join=1

Enter your email address and password that you've used for the node.

I can't login. Nothing happens.

Carlos

PY 222
12-13-2005, 02:04 PM
Originally posted by em99010pepe
I can't login. Nothing happens.

Carlos

I just tried logging into my userpage and its all working ok. I can change Teams and look at my Options.

Maybe you should give the Admin of Majestic12 an email or visit their forums.

em99010pepe
12-13-2005, 02:32 PM
Originally posted by PY 222
I just tried logging into my userpage and its all working ok. I can change Teams and look at my Options.

Maybe you should give the Admin of Majestic12 an email or visit their forums.

Found the problem. The page doesn't work on FireFox. Used IE. Thanks.

Carlos

Bok
12-13-2005, 03:24 PM
Majestic-12 added to DCR, signature and userspage..

ohh, I get 100 points in the DCR for it :) Not for long I suspect!!

Bok

PY 222
12-13-2005, 04:22 PM
Originally posted by Bok
Majestic-12 added to DCR, signature and userspage..

ohh, I get 100 points in the DCR for it :) Not for long I suspect!!

Bok

Just added mine.

Thanks Bok.

Now let me see how many points I'll get. :D

Bok
12-13-2005, 04:27 PM
It'll be a max of 200 as long as there is only 2 of us. You'll take the lions share and I'll go down to 1 or 2 points :)

Till PCZ adds his in of course and others too... :)

Bok

PY 222
12-13-2005, 04:44 PM
Originally posted by Bok
It'll be a max of 200 as long as there is only 2 of us. You'll take the lions share and I'll go down to 1 or 2 points :)

Till PCZ adds his in of course and others too... :)

Bok

Just saw that I took 298.51 and left you with 1.24. Sorrie Bok. :spray: