PennMUSH Community

Ticket #6558 (closed released)

Opened 4 years ago

Chunk allocator problems

Reported by: anonymous Assigned to: devteam
Priority: minor Milestone:
Keywords: Cc:
Visibility: Public

Description

Greetings,


I imagine that you guys have plans to improve this subsystem of the
package.  But, I have been having problems with it for months now and have
seen nothing in the way of the changelog regarding chunk allocator
improvements.  I wonder that perhaps you could use some profile data from a
MUSH that is experiencing reproduceable issues?  I admin on such a game.

The first problem we were experiencing is jittery CPU usage, varying
(according to top) in intervals less than 1 second between never less than
15% and often as much 80%, but averaging 30%-60%.  On occasion, it would
jump up to >90% and subside a few minutes later, but during this time
players cannot send or receive data with the game.  The cause of this was a
mystery for a while, until the server admin became concerned about resource
use and I started an investigation.  I found that the cause was my setting
the chunk_cache_memory option to 2000000000 as indicated by the comments
for that option in mush.cnf to effectively 'incapacitate' the chunk
allocator.  I did this originally to save myself from the headache of other
chunk allocator bugs that I've been introduced to in the past.

As a remedy, I changed the chunk_cache_memory option back to the default,
1000000, and the CPU freakage problem was immediately solved -- now the
game averages at most 0.9% according to top, as I expect it should. 
Granted some would say this is not the most reliable way to examine
performance, but I hope that anyone would agree that a measure of 0.9% is
better than 60%.

However, where the game's CPU usage would jump up and down and occasionally
spike when the chunk_cache_allocator option was set to a really high value,
now it just spikes on occasion (every 5 to 7 days) and never comes back
down!  When this happens, data cannot be sent or received from the game and
the only way to recover is to kill it and start over.  Players have waited
as long as 10 hours for a response from the server before someone with
shell access could kill and restart it.  This is separate mystery from the
first, because I can't actually reproduce this, and now I feel compelled to
predict when it's going to happen.  So far I've noted that it mostly
happens during the weekend (or as last as Monday).  I wonder that it has
something to do with player activity, but I can't be sure.  Average number
of player connections for a weekend evening is 25-35 I suppose.

Aside from actually providing you with some actual information that may
help to fix problems that are related to the chunk allocator, is there a
sort of tool I can use to monitor response times for commands sent to a
MUSH -- one that would support sending me a notification if the response
time is over a certain limit and/or kill and restart the game?  This would
be different from a network monitor tool because there is nothing wrong
with the host or its connection -- the problem is in the game process.


Sholevi (mrdyg@yahoo.com)

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Change History

10/11/04 11:16:33 changed by D Young <mrdyg@yahoo.com>

Here's the specs of the original host from /proc:

OS:  Red Hat Linux 7.3 2.96-126 (RH7), Linux version 2.4.20-28.7
CPU: PIII Coppermmine (Celeron) 930MHz, 256KB cache
RAM: 512MB
FS:  ext3
Swap:  1052216 kB (~1gig)

However, I would like to note that I am able to reproduce the CPU freakage
problem by copying the game to a completely different host.  Although the
exact numbers are different since the specs of the host are different, the
results are similar -- rapidly fluctuating CPU usage no less than 20% and
often as high as 60%, occasional spiking -- and it dissipates when
chunk_cache_memory is set to the default value.

Specs of the other host for testing:

OS:   Red Hat Linux 3.3.3-7 (Fedora Core 2), Linux version 2.6.5-1.358
CPU:  PII 300MHz, 512KB cache
RAM:  128MB
Disk:  ext3
Swap:  256MB




--- Alan Schwartz <dunemush@pennmush.org> wrote:

> Quoting D Young (mrdyg@yahoo.com):
> > 
> > Greetings,
> > 
> > 
> > I imagine that you guys have plans to improve this subsystem of the
> > package.  But, I have been having problems with it for months now and
> have
> > seen nothing in the way of the changelog regarding chunk allocator
> > improvements.  I wonder that perhaps you could use some profile data
> from a
> > MUSH that is experiencing reproduceable issues?  I admin on such a
> game.
> 
> Fixes to the allocator are the last thing holding up the release
> of 1.7.8, so yes, we have plans. And your data will undoubtedly be
> very helpful (particularly as the allocator is not doing these
> nasty things here).
> 
> Could you tell us:
> 
> * Your OS, with as much specificity as possible (e.g., what kernel
>   version?)
> * The filesystem you use (ext2, ext3, bsd, reiserfs, etc.) for the
>   partition with your game directory
> * The CPU, memory, and swap specs of your system
> 
>  
> > Aside from actually providing you with some actual information that may
> > help to fix problems that are related to the chunk allocator, is there
> a
> > sort of tool I can use to monitor response times for commands sent to a
> > MUSH -- one that would support sending me a notification if the
> response
> > time is over a certain limit and/or kill and restart the game?  This
> would
> > be different from a network monitor tool because there is nothing wrong
> > with the host or its connection -- the problem is in the game process.
> 
> Well, you could probably use a bot for this, connected to the server,
> and issuing some periodic commands that will result in known output,
> and timing the result. I don't have one handy, though. Another
> possibility would be a watchdog daemon using a simple expect script
> or something like that.
> 
>  - Javelin
> 
> -- 
>
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Javelin@M*U*S*H, once Paul@DuneMUSH | Alan Schwartz
> <dunemush@pennmush.org>
>    (mush.pennmush.org 4201)         |      PennMUSH Server Maintainer
>
=-------------------------------------------------------------------------=
>    PennMUSH God's Guide: http://www.pennmush.org/~alansz/guide.html
>         PennMUSH Source: http://ftp.pennmush.org/Source
>      PennMUSH Community: http://community.pennmush.org
>
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 



		
_______________________________
Do you Yahoo!?
Declare Yourself - Register online to vote today!
http://vote.yahoo.com

10/11/04 11:33:10 changed by Alan Schwartz <dunemush@pennmush.org>

Quoting D Young (mrdyg@yahoo.com):
> 
> Greetings,
> 
> 
> I imagine that you guys have plans to improve this subsystem of the
> package.  But, I have been having problems with it for months now and have
> seen nothing in the way of the changelog regarding chunk allocator
> improvements.  I wonder that perhaps you could use some profile data from a
> MUSH that is experiencing reproduceable issues?  I admin on such a game.

Fixes to the allocator are the last thing holding up the release
of 1.7.8, so yes, we have plans. And your data will undoubtedly be
very helpful (particularly as the allocator is not doing these
nasty things here).

Could you tell us:

* Your OS, with as much specificity as possible (e.g., what kernel
  version?)
* The filesystem you use (ext2, ext3, bsd, reiserfs, etc.) for the
  partition with your game directory
* The CPU, memory, and swap specs of your system

 
> Aside from actually providing you with some actual information that may
> help to fix problems that are related to the chunk allocator, is there a
> sort of tool I can use to monitor response times for commands sent to a
> MUSH -- one that would support sending me a notification if the response
> time is over a certain limit and/or kill and restart the game?  This would
> be different from a network monitor tool because there is nothing wrong
> with the host or its connection -- the problem is in the game process.

Well, you could probably use a bot for this, connected to the server,
and issuing some periodic commands that will result in known output,
and timing the result. I don't have one handy, though. Another
possibility would be a watchdog daemon using a simple expect script
or something like that.

 - Javelin

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Javelin@M*U*S*H, once Paul@DuneMUSH | Alan Schwartz <dunemush@pennmush.org>
   (mush.pennmush.org 4201)         |      PennMUSH Server Maintainer
=-------------------------------------------------------------------------=
   PennMUSH God's Guide: http://www.pennmush.org/~alansz/guide.html
        PennMUSH Source: http://ftp.pennmush.org/Source
     PennMUSH Community: http://community.pennmush.org
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

10/15/04 10:25:26 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

[doing a little rearranging to keep things in sequence]

In message:  <20041011171640.42FC9F010@tala.mede.uic.edu>
             D Young <mrdyg@yahoo.com> writes:

>--- Alan Schwartz <dunemush@pennmush.org> wrote:
>
>> Quoting D Young (mrdyg@yahoo.com):
>> > 
>> > I imagine that you guys have plans to improve this subsystem of the
>> > package.  But, I have been having problems with it for months now
>> > and have seen nothing in the way of the changelog regarding chunk
>> > allocator improvements.

This is my fault; I have a mostly-working replacement built, but have
not had time to iron out the last few bugs (which unfortunately are
crasher-quality).  The replacement _should_ take care of all known
outstanding issues, but only if it works.

>> > I wonder that perhaps you could use some profile data from a MUSH
>> > that is experiencing reproduceable issues?  I admin on such a game.

Actually, the CPU behaviour you've been experiencing in conjunction
with changing the cache size makes perfect sense to me, and I don't
need profile data to back it up; in short, the migration code uses
more CPU when it has more chunks to move around, and if you have a
very large cache, then in any given call to the migrator, more of
the chunks it has license to move will actually be in memory and
able to move.  The amount of CPU uses scales non-linearly with the
number of chunks, so lots of chunks can get _really_ bad.

>> Fixes to the allocator are the last thing holding up the release
>> of 1.7.8, so yes, we have plans.

Plans which I need to get back to.  Sorry, folks.

>Here's the specs of the original host from /proc:

[snip]

>However, I would like to note that I am able to reproduce the CPU freakage
>problem by copying the game to a completely different host.

Yes, this makes sense to me.  It has to do with your large cache size
and very large game.

For a rather damoclean solution, you can comment out the call to
migrate_stuff() at about line 224 of timer.c.  This will completely
disable chunk migration and should eliminate the continual CPU load,
at the cost of allowing memory to become increasingly fragmented and
unusable over time.  If you decide to do this, I _would_ be interested
in hearing about your fragmentation and saturation rates (in
@stats/chunk, on the line about free chunks and the line about storage),
measured every day or so for an extended run.

Heck, that last is an experiment that would be interesting to run on
M*U*S*H, too.

- Alex

10/15/04 16:12:57 changed by D Young <mrdyg@yahoo.com>

I would like to help participate in such an experiment.  However, this is a
live game with an active playerbase.  The ultimate decision lies with the
head wizard -- however, perhaps I can convince him to participate if you
help me define what you mean by increasing fragmentation of the game in
memory to a point where it is unusable?

I assume this wouldn't work if I copied the game to a test site, with
little to no player activity?  If I could, that would be a safer approach
-- safe from the all the addicted MUSHers that will threaten to slit their
wrists if the game crashes and is unavailable for too long.




--- "T. Alexander Popiel" <popiel@wolfskeep.com> wrote:

> [doing a little rearranging to keep things in sequence]
> 
> In message:  <20041011171640.42FC9F010@tala.mede.uic.edu>
>              D Young <mrdyg@yahoo.com> writes:
> 
> >--- Alan Schwartz <dunemush@pennmush.org> wrote:
> >
> >> Quoting D Young (mrdyg@yahoo.com):
> >> > 
> >> > I imagine that you guys have plans to improve this subsystem of the
> >> > package.  But, I have been having problems with it for months now
> >> > and have seen nothing in the way of the changelog regarding chunk
> >> > allocator improvements.
> 
> This is my fault; I have a mostly-working replacement built, but have
> not had time to iron out the last few bugs (which unfortunately are
> crasher-quality).  The replacement _should_ take care of all known
> outstanding issues, but only if it works.
> 
> >> > I wonder that perhaps you could use some profile data from a MUSH
> >> > that is experiencing reproduceable issues?  I admin on such a game.
> 
> Actually, the CPU behaviour you've been experiencing in conjunction
> with changing the cache size makes perfect sense to me, and I don't
> need profile data to back it up; in short, the migration code uses
> more CPU when it has more chunks to move around, and if you have a
> very large cache, then in any given call to the migrator, more of
> the chunks it has license to move will actually be in memory and
> able to move.  The amount of CPU uses scales non-linearly with the
> number of chunks, so lots of chunks can get _really_ bad.
> 
> >> Fixes to the allocator are the last thing holding up the release
> >> of 1.7.8, so yes, we have plans.
> 
> Plans which I need to get back to.  Sorry, folks.
> 
> >Here's the specs of the original host from /proc:
> 
> [snip]
> 
> >However, I would like to note that I am able to reproduce the CPU
> freakage
> >problem by copying the game to a completely different host.
> 
> Yes, this makes sense to me.  It has to do with your large cache size
> and very large game.
> 
> For a rather damoclean solution, you can comment out the call to
> migrate_stuff() at about line 224 of timer.c.  This will completely
> disable chunk migration and should eliminate the continual CPU load,
> at the cost of allowing memory to become increasingly fragmented and
> unusable over time.  If you decide to do this, I _would_ be interested
> in hearing about your fragmentation and saturation rates (in
> @stats/chunk, on the line about free chunks and the line about storage),
> measured every day or so for an extended run.
> 
> Heck, that last is an experiment that would be interesting to run on
> M*U*S*H, too.
> 
> - Alex
> 



		
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail 

10/15/04 18:01:56 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

In message:  <20041015221257.66266.qmail@web50102.mail.yahoo.com>
             D Young <mrdyg@yahoo.com> writes:

>I would like to help participate in such an experiment.  However, this is a
>live game with an active playerbase.  The ultimate decision lies with the
>head wizard -- however, perhaps I can convince him to participate if you
>help me define what you mean by increasing fragmentation of the game in
>memory to a point where it is unusable?

What it means is that the server process would slowly (or possibly
not-so-slowly, if my guesses are wrong) bloat, eating more and more
memory.  In no case would your database be in danger... it's just
that the game would, over time, not use memory as efficiently.
At worst, the game would end up taking up twice as much memory as it
ideally might... and more likely, it'd limit itself to about half-again
as much.

As attributes are destroyed, the memory they used would appear as
holes that could be reused... but if those holes were reused by
slightly smaller attributes, then the remainder little bits of memory
would likely be too small for much of anything.  As time progressed,
this fragmentation of memory into little bits would become pervasive,
limiting the density of actual allocations.

The migrate_stuff() call that you'd be disabling is a process which
(among other things) gathers up these little bits of memory and
coalesces them into larger pieces that can be more effectively used.

>I assume this wouldn't work if I copied the game to a test site, with
>little to no player activity?  If I could, that would be a safer approach
>-- safe from the all the addicted MUSHers that will threaten to slit their
>wrists if the game crashes and is unavailable for too long.

You are correct that running this modification on a test site won't
provide us with useful data.  However, doing this experiment would not
open you up to server crashes... and should completely eliminate the
hangs that you reported earlier.

- Alex

10/15/04 22:56:44 changed by D Young <mrdyg@yahoo.com>

Okay!  I will implement this change, and will try to forward the results
and @stats/chunk output to you approximately once a week (summarizing each
of 7 days).




--- "T. Alexander Popiel" <popiel@wolfskeep.com> wrote:

> In message:  <20041015221257.66266.qmail@web50102.mail.yahoo.com>
>              D Young <mrdyg@yahoo.com> writes:
> 
> >I would like to help participate in such an experiment.  However, this
> is a
> >live game with an active playerbase.  The ultimate decision lies with
> the
> >head wizard -- however, perhaps I can convince him to participate if you
> >help me define what you mean by increasing fragmentation of the game in
> >memory to a point where it is unusable?
> 
> What it means is that the server process would slowly (or possibly
> not-so-slowly, if my guesses are wrong) bloat, eating more and more
> memory.  In no case would your database be in danger... it's just
> that the game would, over time, not use memory as efficiently.
> At worst, the game would end up taking up twice as much memory as it
> ideally might... and more likely, it'd limit itself to about half-again
> as much.
> 
> As attributes are destroyed, the memory they used would appear as
> holes that could be reused... but if those holes were reused by
> slightly smaller attributes, then the remainder little bits of memory
> would likely be too small for much of anything.  As time progressed,
> this fragmentation of memory into little bits would become pervasive,
> limiting the density of actual allocations.
> 
> The migrate_stuff() call that you'd be disabling is a process which
> (among other things) gathers up these little bits of memory and
> coalesces them into larger pieces that can be more effectively used.
> 
> >I assume this wouldn't work if I copied the game to a test site, with
> >little to no player activity?  If I could, that would be a safer
> approach
> >-- safe from the all the addicted MUSHers that will threaten to slit
> their
> >wrists if the game crashes and is unavailable for too long.
> 
> You are correct that running this modification on a test site won't
> provide us with useful data.  However, doing this experiment would not
> open you up to server crashes... and should completely eliminate the
> hangs that you reported earlier.
> 
> - Alex
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

10/20/04 20:54:19 changed by D Young <mrdyg@yahoo.com>

On the first day (Friday Oct 16), I commented out the call to migrate_stuff()
in timer.c and did @shutdown/reboot.  This was the output from @stats/chunk. 
I would like to note that @stats/chunk was polled at different times of each
day, and reading @stats/chunk at different times on any given day returned
different results that I suppose correlate to player activity.  Hope this is
useful.


Friday 16th:

Chunks:         54743 allocated (   7315628 bytes,     122026 ( 1%) overhead)
                  42203 short     (    576230 bytes,      84406 (14%)
overhead)
                  12540 medium    (   6739398 bytes,      37620 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                   35 free      (     19028 bytes,       2307 (12%)
fragmented)
Regions:          112 total,       16 cached
Storage:      7338400 total (99% saturation)
Paging:          3350 out,       3254 in

Period:             0 (     50460 accesses so far,          8 chunks at max)
Activity:       56632 creates,       1889 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact





Saturday 16th:

Chunks:         54751 allocated (   7319006 bytes,     122058 ( 1%) overhead)
                  42195 short     (    574233 bytes,      84390 (14%)
overhead)
                  12556 medium    (   6744773 bytes,      37668 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                  515 free      (     81138 bytes,      21625 (26%)
fragmented)
Regions:          113 total,       16 cached
Storage:      7403900 total (98% saturation)
Paging:        238798 out,     238701 in

Period:             0 (   3913364 accesses so far,        503 chunks at max)
Activity:      205273 creates,     150522 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact




Sunday 17th:

Chunks:         54647 allocated (   7329939 bytes,     121852 ( 1%) overhead)
                  42089 short     (    575539 bytes,      84178 (14%)
overhead)
                  12558 medium    (   6754400 bytes,      37674 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                  808 free      (     70205 bytes,      33395 (47%)
fragmented)
Regions:          113 total,       16 cached
Storage:      7403900 total (99% saturation)
Paging:        485609 out,     485512 in

Period:             0 (   7413659 accesses so far,        755 chunks at max)
Activity:      367418 creates,     312771 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact




Monday 18th:

Chunks:         54822 allocated (   7345392 bytes,     122217 ( 1%) overhead)
                  42249 short     (    577106 bytes,      84498 (14%)
overhead)
                  12573 medium    (   6768286 bytes,      37719 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                  869 free      (    120240 bytes,      37294 (31%)
fragmented)
Regions:          114 total,       16 cached
Storage:      7469400 total (98% saturation)
Paging:        632277 out,     632179 in

Period:             0 (   9375620 accesses so far,        920 chunks at max)
Activity:      444214 creates,     389392 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact




Teusday 19:

Chunks:         54938 allocated (   7358059 bytes,     122472 ( 1%) overhead)
                  42342 short     (    577783 bytes,      84684 (14%)
overhead)
                  12596 medium    (   6780276 bytes,      37788 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                  935 free      (    107573 bytes,      47772 (44%)
fragmented)
Regions:          114 total,       16 cached
Storage:      7469400 total (98% saturation)
Paging:        850343 out,     850245 in

Period:             0 (  12558815 accesses so far,       1094 chunks at max)
Activity:      623460 creates,     568522 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact



Wednesday 20th:

Chunks:         55127 allocated (   7364436 bytes,     122864 ( 1%) overhead)
                  42517 short     (    580554 bytes,      85034 (14%)
overhead)
                  12610 medium    (   6783882 bytes,      37830 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                  990 free      (    101196 bytes,      53757 (53%)
fragmented)
Regions:          114 total,       16 cached
Storage:      7469400 total (98% saturation)
Paging:       1189678 out,    1189580 in

Period:             0 (  17506883 accesses so far,       1380 chunks at max)
Activity:      876684 creates,     821557 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact





--- "T. Alexander Popiel" <popiel@wolfskeep.com> wrote:

> In message:  <20041015221257.66266.qmail@web50102.mail.yahoo.com>
>              D Young <mrdyg@yahoo.com> writes:
> 
> >I would like to help participate in such an experiment.  However, this
> is a
> >live game with an active playerbase.  The ultimate decision lies with
> the
> >head wizard -- however, perhaps I can convince him to participate if you
> >help me define what you mean by increasing fragmentation of the game in
> >memory to a point where it is unusable?
> 
> What it means is that the server process would slowly (or possibly
> not-so-slowly, if my guesses are wrong) bloat, eating more and more
> memory.  In no case would your database be in danger... it's just
> that the game would, over time, not use memory as efficiently.
> At worst, the game would end up taking up twice as much memory as it
> ideally might... and more likely, it'd limit itself to about half-again
> as much.
> 
> As attributes are destroyed, the memory they used would appear as
> holes that could be reused... but if those holes were reused by
> slightly smaller attributes, then the remainder little bits of memory
> would likely be too small for much of anything.  As time progressed,
> this fragmentation of memory into little bits would become pervasive,
> limiting the density of actual allocations.
> 
> The migrate_stuff() call that you'd be disabling is a process which
> (among other things) gathers up these little bits of memory and
> coalesces them into larger pieces that can be more effectively used.
> 
> >I assume this wouldn't work if I copied the game to a test site, with
> >little to no player activity?  If I could, that would be a safer
> approach
> >-- safe from the all the addicted MUSHers that will threaten to slit
> their
> >wrists if the game crashes and is unavailable for too long.
> 
> You are correct that running this modification on a test site won't
> provide us with useful data.  However, doing this experiment would not
> open you up to server crashes... and should completely eliminate the
> hangs that you reported earlier.
> 
> - Alex
> 











		
_______________________________
Do you Yahoo!?
Declare Yourself - Register online to vote today!
http://vote.yahoo.com

10/21/04 00:09:22 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

In message:  <20041021025419.17731.qmail@web50110.mail.yahoo.com>
             D Young <mrdyg@yahoo.com> writes:
>
>On the first day (Friday Oct 16), I commented out the call to migrate_stuff()
>in timer.c and did @shutdown/reboot.  This was the output from @stats/chunk. 
>I would like to note that @stats/chunk was polled at different times of each
>day, and reading @stats/chunk at different times on any given day returned
>different results that I suppose correlate to player activity.  Hope this is
>useful.

Yes, quite useful.  Doing it at different times each day isn't a big
deal; it'll even out overall.  The game's behaving better than I thought
it would...  so far you haven't seen significant degredation in the
saturation rate (overall memory efficiency)... it's still at 98%.  The
fragmentation rate is climbing dramatically, though, as I though it
would... but that doesn't seem to be hurting much, yet.  And it's not
good because nothing is happening... you've got over 100,000 creates
and deletes each day.

In addition to the @stats/chunk, could you also do @stats/region?
I'd like to see if the distribution of chunks is suffering, or if
it's staying fairly well sorted...

If things stay this good for a couple weeks, then I may just bag
the whole migration thing entirely as unnecessary.  Hrm.

- Alex

10/21/04 18:58:46 changed by D Young <mrdyg@yahoo.com>

Here's a capture of @stats/chunk/region Thursday.  Also, I wonder what it
means when a different MUSH that has implemented the chunk allocator has a
much lower saturation rate (<20%).


Chunks:         54802 allocated (   7291925 bytes,     122133 ( 1%) overhead)
                  42273 short     (    577786 bytes,      84546 (14%)
overhead)
                  12529 medium    (   6714139 bytes,      37587 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 1304 free      (    173707 bytes,      78112 (44%)
fragmented)
Regions:          114 total,       16 cached
Storage:      7469400 total (97% saturation)
Paging:       1537077 out,    1536979 in

Period:             0 (  22063570 accesses so far,       1640 chunks at max)
Activity:     1107305 creates,    1052503 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact



Chart number of regions (y) vs. references (x)
    32 |  *
       |  *
       |  *
       |  *
       |  *
       |  **
       |  **
       |  **
       |  **
       |  ***
       |  ***
       |  ***
       |  ***
       |  ***
       | ****
       | *****
       | ******
       | ******
       | ******
     0 | *******   *   *
       +----------------------------------------------------------------
        0                              |                             255





--- "T. Alexander Popiel" <popiel@wolfskeep.com> wrote:

> In message:  <20041021025419.17731.qmail@web50110.mail.yahoo.com>
>              D Young <mrdyg@yahoo.com> writes:
> >
> >On the first day (Friday Oct 16), I commented out the call to
> migrate_stuff()
> >in timer.c and did @shutdown/reboot.  This was the output from
> @stats/chunk. 
> >I would like to note that @stats/chunk was polled at different times of
> each
> >day, and reading @stats/chunk at different times on any given day returned
> >different results that I suppose correlate to player activity.  Hope this
> is
> >useful.
> 
> Yes, quite useful.  Doing it at different times each day isn't a big
> deal; it'll even out overall.  The game's behaving better than I thought
> it would...  so far you haven't seen significant degredation in the
> saturation rate (overall memory efficiency)... it's still at 98%.  The
> fragmentation rate is climbing dramatically, though, as I though it
> would... but that doesn't seem to be hurting much, yet.  And it's not
> good because nothing is happening... you've got over 100,000 creates
> and deletes each day.
> 
> In addition to the @stats/chunk, could you also do @stats/region?
> I'd like to see if the distribution of chunks is suffering, or if
> it's staying fairly well sorted...
> 
> If things stay this good for a couple weeks, then I may just bag
> the whole migration thing entirely as unnecessary.  Hrm.
> 
> - Alex
> 



		
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail 

10/22/04 09:53:12 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

In message:  <20041022005850.C6622F018@tala.mede.uic.edu>
             D Young <mrdyg@yahoo.com> writes:

>Here's a capture of @stats/chunk/region Thursday.  Also, I wonder what it
>means when a different MUSH that has implemented the chunk allocator has a
>much lower saturation rate (<20%).

It means that that MUSH isn't using memory as efficiently as yours.
There are many possible explanations.  Very small games _can't_ have
a high saturation rate (the regions are each fixed at 64K, so if
you have only 8K of data, you'll have about a 12% saturation in
your single region; similar for games with only a couple regions).
Also, there's a known problem (which has been delaying the release of
1.7.8) where the current version of the migrator gradually spreads
stuff out much more than necessary (so if a game hasn't rebooted in
a few months, then it'll have a low saturation rate).  Etc.

>Chart number of regions (y) vs. references (x)
>    32 |  *
>       |  *
>       |  *
>       |  *
>       |  *
>       |  **
>       |  **
>       |  **
>       |  **
>       |  ***
>       |  ***
>       |  ***
>       |  ***
>       |  ***
>       | ****
>       | *****
>       | ******
>       | ******
>       | ******
>     0 | *******   *   *
>       +----------------------------------------------------------------
>        0                              |                             255

That's very interesting... your largest spike is very close to 0...
have you changed the dump interval from its default of one hour to
something longer?

- Alex

10/22/04 10:07:29 changed by D Young <mrdyg@yahoo.com>

--- "T. Alexander Popiel" <popiel@wolfskeep.com> wrote:
> That's very interesting... your largest spike is very close to 0...
> have you changed the dump interval from its default of one hour to
> something longer?


Nope.  Forking dumps still occur once every hour.

As a positive side note, the game process has been up for a week now without
freezing and requiring kill -9 -- but you did anticipate that.



		
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail 

10/22/04 15:53:10 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

In message:  <20041022160729.19776.qmail@web50104.mail.yahoo.com>
             D Young <mrdyg@yahoo.com> writes:
>
>--- "T. Alexander Popiel" <popiel@wolfskeep.com> wrote:
>> That's very interesting... your largest spike is very close to 0...
>> have you changed the dump interval from its default of one hour to
>> something longer?
>
>Nope.  Forking dumps still occur once every hour.

Ah ha!  Forking dumps won't push the spike up in the same way that
non-forking dumps do.  I hadn't thought about that scenario.  Shouldn't
hurt anything, just wasn't what I was expecting.

>As a positive side note, the game process has been up for a week now without
>freezing and requiring kill -9 -- but you did anticipate that.

Good to hear that my theory has borne out in practice.

- Alex

10/30/04 09:33:33 changed by D Young <mrdyg@yahoo.com>

Greetings,
   Some more @stats/chunk and @stats/region logs.  :-)







Saturday Oct 23:

Chunks:         54946 allocated (   7322627 bytes,     122447 ( 1%) overhead)
                  42391 short     (    580264 bytes,      84782 (14%)
overhead)
                  12555 medium    (   6742363 bytes,      37665 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 1425 free      (    143005 bytes,      84773 (59%)
fragmented)
Regions:          114 total,       16 cached
Storage:      7469400 total (98% saturation)
Paging:       1959805 out,    1959707 in

Period:             0 (  27910145 accesses so far,       1965 chunks at max)
Activity:     1406306 creates,    1351360 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact


Chart number of regions (y) vs. references (x)
    26 |   *
       |   *
       |   *
       |  **
       |  ***
       |  ***
       |  ***
       |  ***
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  ******
       |  ******
       |  *******       *
     0 | ********       *
       +----------------------------------------------------------------
        0                              |                             255







Late Sat Night Oct 23:

Chunks:         54973 allocated (   7327282 bytes,     122516 ( 1%) overhead)
                  42403 short     (    580433 bytes,      84806 (14%)
overhead)
                  12570 medium    (   6746849 bytes,      37710 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 1517 free      (    138350 bytes,      86014 (62%)
fragmented)
Regions:          114 total,       16 cached
Storage:      7469400 total (98% saturation)
Paging:       2046095 out,    2045997 in

Period:             0 (  29146946 accesses so far,       2049 chunks at max)
Activity:     1464185 creates,    1409212 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    26 |    *
       |    *
       |    *
       |    *
       |   **
       |   **
       |   **
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  ****
       |  *****
       |  *****
       |  ******
       |  ******
       |  *******       *
     0 | ******** * *   *
       +----------------------------------------------------------------
        0                              |                             255






Sunday Night Oct 24:

Chunks:         55194 allocated (   7366470 bytes,     122994 ( 1%) overhead)
                  42588 short     (    583469 bytes,      85176 (14%)
overhead)
                  12606 medium    (   6783001 bytes,      37818 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 1536 free      (    164650 bytes,      79843 (48%)
fragmented)
Regions:          115 total,       16 cached
Storage:      7534900 total (97% saturation)
Paging:       2281794 out,    2281695 in

Period:             0 (  32400695 accesses so far,       2212 chunks at max)
Activity:     1617074 creates,    1561880 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    29 |    *
       |    *
       |    *
       |    *
       |    *
       |    *
       |    *
       |   **
       |   ***
       |   ***
       |   ***
       |   ***
       |  ****
       |  *****
       |  *****
       |  *****
       |  ***** *
       |  *******
       | ********       *
     0 | ********   *   *
       +----------------------------------------------------------------
        0                              |                             255





Late Monday Night Oct 25:

Chunks:         55168 allocated (   7387395 bytes,     122958 ( 1%) overhead)
                  42546 short     (    583402 bytes,      85092 (14%)
overhead)
                  12622 medium    (   6803993 bytes,      37866 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 1692 free      (    143725 bytes,      82068 (57%)
fragmented)
Regions:          115 total,       16 cached
Storage:      7534900 total (98% saturation)
Paging:       2463539 out,    2463440 in

Period:             0 (  34998819 accesses so far,       2358 chunks at max)
Activity:     1748692 creates,    1693524 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    26 |    *
       |    *
       |    *
       |    *
       |   **
       |   **
       |   **
       |   **
       |   **
       |   ** *
       |   ****
       |   ****
       |   ****
       |   ****
       |  ******
       |  ******
       |  *******
       |  ********
       | *********
     0 | *********  *   *
       +----------------------------------------------------------------
        0                              |                             255






Teusday Oct 26:


Chunks:         55290 allocated (   7405643 bytes,     123240 ( 1%) overhead)
                  42630 short     (    584046 bytes,      85260 (14%)
overhead)
                  12660 medium    (   6821597 bytes,      37980 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 1877 free      (    125477 bytes,      87342 (69%)
fragmented)
Regions:          115 total,       16 cached
Storage:      7534900 total (98% saturation)
Paging:       2680741 out,    2680642 in

Period:             0 (  38193722 accesses so far,       2562 chunks at max)
Activity:     1882693 creates,    1827403 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    25 |    *
       |    *
       |    *
       |    *
       |    *
       |   **
       |   **
       |   ***
       |   ****
       |   ****
       |   ****
       |   ****
       |   ****
       |   ****
       |   ******
       |   ******
       |  *******
       |  ********
       |  ********
     0 |  *********  *   *
       +----------------------------------------------------------------
        0                              |                             255





Wednesday Oct 28:

Chunks:         54122 allocated (   7407765 bytes,     120925 ( 1%) overhead)
                  41441 short     (    578258 bytes,      82882 (14%)
overhead)
                  12681 medium    (   6829507 bytes,      38043 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 2992 free      (    188843 bytes,      95928 (50%)
fragmented)
Regions:          116 total,       16 cached
Storage:      7600400 total (97% saturation)
Paging:       2866793 out,    2866693 in

Period:             0 (  40743015 accesses so far,       2725 chunks at max)
Activity:     2019158 creates,    1965036 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    24 |     *
       |     *
       |     *
       |    **
       |    **
       |    **
       |    **
       |    **
       |    **
       |   ***
       |   ****
       |   ****
       |   **** *
       |   ******
       |   ******
       |   ******
       |  ********
       |  ********
       |  *********
     0 |  **********     *
       +----------------------------------------------------------------
        0                              |                             255




Early Friday Morning Oct 29:

Chunks:         54151 allocated (   7403319 bytes,     120965 ( 1%) overhead)
                  41488 short     (    578583 bytes,      82976 (14%)
overhead)
                  12663 medium    (   6824736 bytes,      37989 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 3043 free      (    193289 bytes,     102832 (53%)
fragmented)
Regions:          116 total,       16 cached
Storage:      7600400 total (97% saturation)
Paging:       3077935 out,    3077835 in

Period:             0 (  43528575 accesses so far,       2911 chunks at max)
Activity:     2181710 creates,    2127559 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    25 |     *
       |     *
       |     *
       |     *
       |     *
       |     *
       |     *
       |    **
       |   ****
       |   ****
       |   ****
       |   ****
       |   *****
       |   ******
       |   ******
       |   ******
       |   ******  *
       |   ******  *
       |   *********
     0 |  **********  *   *
       +----------------------------------------------------------------
        0                              |                             255




Saturday Morning Oct 30:

Chunks:         54290 allocated (   7412939 bytes,     121262 ( 1%) overhead)
                  41608 short     (    582160 bytes,      83216 (14%)
overhead)
                  12682 medium    (   6830779 bytes,      38046 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 3418 free      (    183669 bytes,     110879 (60%)
fragmented)
Regions:          116 total,       16 cached
Storage:      7600400 total (97% saturation)
Paging:       3286827 out,    3286727 in

Period:             0 (  46223371 accesses so far,       3042 chunks at max)
Activity:     2317339 creates,    2263049 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    25 |     *
       |     *
       |     *
       |     *
       |     *
       |     *
       |     **
       |     **
       |    ***
       |    ***
       |    ***
       |   ****
       |   ****
       |   **** *
       |   ******
       |   *******
       |   ******* *
       |  ******** *
       |  **********
     0 |  **********  *   *
       +----------------------------------------------------------------
        0                              |                             255


		
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail 

10/30/04 13:13:45 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

In message:  <20041030153334.18903.qmail@web50110.mail.yahoo.com>
             D Young <mrdyg@yahoo.com> writes:
>Greetings,
>   Some more @stats/chunk and @stats/region logs.  :-)

Thank you very much.

I think that this shows pretty conclusively that we don't need migration
for maintaining a high saturation rate.  That means that the primary value
of migration is sorting for improved locality of reference, and that's a
much lower priority.  The improved locality of reference could reduce
the ~200K paging operations that it's doing a day... but really,
approximately 1.5 gigabytes of disk I/O per day (148 kilobytes a second,
on average) is not a huge thing to worry about, particularly when you
take the OS's buffercache into account.

I've got a seriously stripped-down version of migration that I'll be ready
to beta on M*U*S*H soon... just as soon as I figure out why it's going
into an infinite loop in walking a free list...

- Alex

11/07/04 18:00:47 changed by D Young <mrdyg@yahoo.com>

More @stats logs in case you need.  I missed a couple of days, but at this
step, I think they are insignificant.

This MUSH will reach 30 days uptime without a single @shutdown or
@shutdown/reboot in a matter of 7 days.




Late Saturday Oct 30:

Chunks:         54408 allocated (   7435919 bytes,     121528 ( 1%) overhead)
                  41696 short     (    584390 bytes,      83392 (14%)
overhead)
                  12712 medium    (   6851529 bytes,      38136 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 3467 free      (    160689 bytes,     112633 (70%)
fragmented)
Regions:          116 total,       16 cached
Storage:      7600400 total (97% saturation)
Paging:       3499829 out,    3499729 in

Period:             0 (  49092922 accesses so far,       3196 chunks at max)
Activity:     2422555 creates,    2368147 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    25 |     *
       |     *
       |     *
       |     *
       |     *
       |     *
       |    **
       |    ***
       |    ***
       |    ***
       |    ***
       |    ***
       |    ***
       |    *** *
       |   *******
       |   ******* *
       |   ******* *
       |   *********
       |   *********
     0 |  **********  *
       +----------------------------------------------------------------
        0                              |                             255




Teusday Nov 02:

Chunks:         54627 allocated (   7472226 bytes,     122007 ( 1%) overhead)
                  41874 short     (    588330 bytes,      83748 (14%)
overhead)
                  12753 medium    (   6883896 bytes,      38259 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 3823 free      (    189870 bytes,     143162 (75%)
fragmented)
Regions:          117 total,       16 cached
Storage:      7665900 total (97% saturation)
Paging:       4194905 out,    4194804 in

Period:             0 (  58189531 accesses so far,       3641 chunks at max)
Activity:     2795720 creates,    2741093 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    27 |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |    * **
       |    * **
       |    ****
       |    ****
       |    ****
       |    ****    *
       |    ****    *
       |    ***** * *
       |    ***** * *
       |   ********** *
     0 | * ************
       +----------------------------------------------------------------
        0                              |                             255




Thursday Nov 04:

Chunks:         54726 allocated (   7477444 bytes,     122231 ( 1%) overhead)
                  41947 short     (    589729 bytes,      83894 (14%)
overhead)
                  12779 medium    (   6887715 bytes,      38337 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 3861 free      (    250140 bytes,     153978 (61%)
fragmented)
Regions:          118 total,       16 cached
Storage:      7731400 total (96% saturation)
Paging:       4592107 out,    4592005 in

Period:             0 (  63582396 accesses so far,       3862 chunks at max)
Activity:     3030782 creates,    2976056 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    26 |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |     **
       |     ***
       |     ***
       |     ***
       |     ***
       |    *****
       |    *****
       |    *****   *
       |    *****   *
       |    *****  ****
       |   ************
     0 | * *************      *
       +----------------------------------------------------------------
        0                              |                             255



Friday Nov 5:

Chunks:         54809 allocated (   7509565 bytes,     122428 ( 1%) overhead)
                  41999 short     (    590574 bytes,      83998 (14%)
overhead)
                  12810 medium    (   6918991 bytes,      38430 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 4001 free      (    218019 bytes,     160805 (73%)
fragmented)
Regions:          118 total,       16 cached
Storage:      7731400 total (97% saturation)
Paging:       4737417 out,    4737315 in

Period:             0 (  65552539 accesses so far,       3922 chunks at max)
Activity:     3123375 creates,    3068566 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    20 |       *
       |     * *
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ****
       |    *****
       |    ******
       |    ******   *
       |    ****** * *
       |    ****** * **
       |    ************
       | * *************  *   *
     0 | **************** *** *                 *
       +----------------------------------------------------------------
        0                              |                             255



Saturday Nov 6:

Chunks:         54958 allocated (   7530724 bytes,     122762 ( 1%) overhead)
                  42112 short     (    593743 bytes,      84224 (14%)
overhead)
                  12846 medium    (   6936981 bytes,      38538 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 4196 free      (    262348 bytes,     156590 (59%)
fragmented)
Regions:          119 total,       16 cached
Storage:      7796900 total (96% saturation)
Paging:       4932796 out,    4932693 in

Period:             0 (  68269178 accesses so far,       4002 chunks at max)
Activity:     3259453 creates,    3204495 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    22 |      *
       |      *
       |      *
       |      *
       |      **
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ***
       |     ****
       |     ****
       |     *****
       |    ******  *  *
       |    ******  *  *
       |    ******* ** *
       |    ************
     0 | * *************   *  *
       +----------------------------------------------------------------
        0                              |                             255



Sunday Nov 07:

Chunks:         54930 allocated (   7529105 bytes,     122712 ( 1%) overhead)
                  42078 short     (    591474 bytes,      84156 (14%)
overhead)
                  12852 medium    (   6937631 bytes,      38556 ( 0%)
overhead)
                      0 long      (         0 bytes,          0 ( 0%)
overhead)
                 4274 free      (    263967 bytes,     171019 (64%)
fragmented)
Regions:          119 total,       16 cached
Storage:      7796900 total (96% saturation)
Paging:       5074806 out,    5074703 in

Period:             0 (  70260337 accesses so far,       4072 chunks at max)
Activity:     3368707 creates,    3313777 deletes this period
Migration:          0 moves this period
                      0 slide             0 fill exact
                      0 away              0 fill inexact
Chart number of regions (y) vs. references (x)
    22 |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |      *
       |     **
       |     ***
       |     ****
       |     ****
       |     ****
       |     ****
       |     *****
       |    ******       |    ******
       |    *******  ***
       |    *******  ***
       |   ******** ****    *
     0 |   **************   * *
       +----------------------------------------------------------------
        0                              |                             255


		
__________________________________ 
Do you Yahoo!? 
Check out the new Yahoo! Front Page. 
www.yahoo.com 
 


11/08/04 12:13:48 changed by "T. Alexander Popiel" <popiel@wolfskeep.com>

In message:  <20041108020047.90107.qmail@web50105.mail.yahoo.com>
             D Young <mrdyg@yahoo.com> writes:

>More @stats logs in case you need.  I missed a couple of days, but at this
>step, I think they are insignificant.

Yeah, the missing days are pretty insignificant.  It seems that there
is a slow degredation of saturation by about 1% every 1-2 weeks.  It
also seems that there is a lot of noise in the number of page operations,
but it's on the order of 100-200k page in/out per day (or about 2 per
second).

I think that's about all we're going to glean from this test; you can stop
collecting data now.  Thank you for doing it.

>This MUSH will reach 30 days uptime without a single @shutdown or
>@shutdown/reboot in a matter of 7 days.

It's unfortunate that this is remarkable. ;-)

- Alex

11/30/04 21:19:50 changed by

Resolved with new allocator, I hope.