Server status

[guru]

Is there anyway you could put a server load vs time graph online so we can see when the best time of the day would be to upload? Like when the server isn't so busy.

guru

[gopher_yarrowzoo]

yeah but then people would start to up load then - maybe we should restrict people to after 5pm say 5pm to midnight Local

[pfb]

or sort out any underlying issues

[guru]

Yea, so? If they upload when the servers aren't as busy, then that will help reduce the load on the server when it is busy.

guru

[IronBits]

Load balancing who'da thunk?
I think that would be a great concept myself to create hourly graphs of bandwidth usage so we the users can find an idle time to get stuff uploaded...

Someone should mention it to Howard and company, I know there are easy packages and free that will do that with minimal configuration...

[Stardragon]

Load balancing?
Of course we've had load balancing since the project started. As I mentioned before, the issue is with the hardware of the database machine.

We will likely post the requirements for any possible contirbutions you are keen to make after I discuss and finalize them with Howard.

[willy1]

You have never said which hardware on the database HP9000 is the problem.

If you know it's hardware, then you should know what part of the hardware. CPU? Memory? Disk? I/O channel? Network connections to the front-end server(s)?

Are you using GlancePlus to watch all your hardware performance metrics? How about kernel parameters? Using up all available of one particular system table?


How many database inserts per hour are you capable of, when the server is running flat out (as it obviously is these days)?

[Brian the Fist]

Originally posted by willy1
You have never said which hardware on the database HP9000 is the problem.

If you know it's hardware, then you should know what part of the hardware. CPU? Memory? Disk? I/O channel? Network connections to the front-end server(s)?

Are you using GlancePlus to watch all your hardware performance metrics? How about kernel parameters? Using up all available of one particular system table?


How many database inserts per hour are you capable of, when the server is running flat out (as it obviously is these days)?

We are running some of the Glance tools and so forth but honestly are not really familiar with them (dont have time to fold proteins, code client AND become full HP-UX administrator ). Anyhow a simple top shows 100% CPU usage when the machine was maxxed out so the bottleneck was probably not the disk or I/O but just DB2 hogging the CPUs as it was doing the inserts. We have not explicitly timed the inserts/minute but could estimate it - does anybody know what a reasonable number would be for a 8-CPU box with 440MHz processors and a decent RAID attached?

[willy1]

I believe I quoted that number in another thread. 50,000 OLTP transactions per minute.

Which processes are using the most CPU (as shown in top)?

[Brian the Fist]

db2agent, as I metioned elsewhere - it was DB2 doing inserts that was the bottleneck.

[guru]

Does db2 take advantage of all 8 processors or does it only bind up to one?

guru

[willy1]

I would hope that it can be configured to use all 8, in one way or another. Wouldn't be much of an enterprise class database engine otherwise.

Since my Informix databases may end up migrating to DB2 if IBM decides to migrate the product (IBM bought Informix a couple of years ago) , I had better start getting a DB2 edumacation.

willy1

[Brian the Fist]

Of course it uses all 8 CPUs, we are not that dumb

[guru]

DB2 V5 incorporates a number of performance enhancement features; many of these aim to exploit multiprocessor architectures. It employs a technique called intraquery parallelism, where complex queries are decomposed into subqueries that are executed in parallel on multiple processors. This suits symmetrical multiprocessor (SMP) architectures, where a number of agents can process the query on shared disks and in shared memory. Massively parallel processor (MPP) and cluster architectures are exploited through hash-based table partitioning or through databases partitioned across clusters. Other improvements aimed at multiprocessor architectures include parallel I/O, parallel index creation, parallel load, and parallel backup and restore utilities.

What kind of disk storage system are you using on this system?

guru

[AMD_is_logical]

Originally posted by guru
What kind of disk storage system are you using on this system?

guru

See the post by Brian the Fist near the end of this thread:
http://www.free-dc.org/forum/showthr...0&pagenumber=2

[guru]

Maybe it's not a hardware issue but a software issue?

Benchmark
guru

[HaloJones]

guru, you can't compare DB2 running on a 4-cpu Intel box running W2KAS with other DBMS and then say this means DB2 on an 8-way box running Unix is therefore also a bad DBMS.

1) Intel 4-way boxes do not have an efficient usage of the 3rd and 4th processor where an N-class HP running HPUX will use all eight in a more efficient (not necessarily efficient but merely more efficient manner)

2) DB2 is an awesome database when used on enterprise class hardware (i.e. anything but Intel) in particular on mainframes

3) DB2 performed badly in those tests because it was being driven by WebLogic using JDBC not because the DB2 was bad itself, more that the drivers for the middleware were bad.


I'm not saying that DB2 is better or worse than MySQL or whatever, simply that that test is fundamentally flawed.

[guru]

It wasn't the overall numbers that I looked at. It's how its performance dropped in half at the 600 user point. It should level off not drop like that. It's kind of what we have seen happening here. It screams up till a point then it takes a nose dive in speed. Once it starts to slow down it crawls. It's either fast or very slow. There is no middle ground.

guru

[Helix_Von_Smelix]

I must be havin' a bad day, switched over to SETI for a few hours, until this blows over. Now their server is down too

well just a wee bit