trinity-users@lists.pearsoncomputing.net

Message: previous - next
Month: December 2019

Re: [BULK] [trinity-users] Re: [BULK] [trinity-users] Re: [BULK] Re: [trinity-users] Re: [BULK] Re: [trinity-users] kmail forever indexing is back

From: Gene Heskett <gheskett@...>
Date: Sat, 14 Dec 2019 08:17:51 -0500
On Saturday 14 December 2019 01:47:01 Gene Heskett wrote:

> On Saturday 14 December 2019 01:38:27 Gene Heskett wrote:
> > On Saturday 14 December 2019 00:27:53 William Morder via
> > trinity-users
> >
> > wrote:
> > > On Friday 13 December 2019 19:49:35 Gene Heskett wrote:
> > > > On Friday 13 December 2019 16:05:18 William Morder via
> > > > trinity-users
> > > >
> > > > wrote:
> > > > > On Friday 13 December 2019 11:52:09 Gene Heskett wrote:
> > > > > > Greetings all;
> > > > > >
> > > > > > So I picked the largest cur dirs, and using mc have moved
> > > > > > about 2 years worth of each to an outside of the ~/Mail view
> > > > > > of kmail. And nuked those index files, so it has to rebuild
> > > > > > them. Its not done with that yet, but we'll see.  Damn this
> > > > > > is getting old.
> > > > > >
> > > > > > Cheers, Gene Heskett
> > > > >
> > > > > The complete corpus of my Kmail (including nearly 20 years of
> > > > > archives like yourself) is only about 4.5 gb, and my machine
> > > > > is built out of spare parts, most of which are older than
> > > > > Utnapishtim's Flood. And while my machine is a little slow
> > > > > sometimes, I never experience anything like you keep
> > > > > describing.
> > > >
> > > > I started a year ago with over 12GB but my moving of older stuff
> > > > out of the Mail directory has now reduced the "du -h Mail" to
> > > > about 4.8GB. I've stopped kmail 4 or 5 times, but on the
> > > > restart, and that nearly always crashes once, but before it
> > > > crashes, maybe 3 seconds elapsed, it has pulled all the "index"
> > > > files out of cache someplace I haven't found.  So I nuked ALL
> > > > the index files 3 times.  Then it crashed about 2-3 seconds
> > > > after startup each time.
> > > >
> > > > A 4th or 5th restart has not crashed, and it updated the dates
> > > > on ALL index files twice, then has shut that off again. The
> > > > largest remaining directory  now is "sent-mail" and its way
> > > > bigger than any of the rest by a factor of at least 2.  So
> > > > momentarily, its behaving itself. 2 hours? a week, a year, the
> > > > rapture? who knows???
> > > >
> > > > Its a bit like asking, while concrete is being poured, if it
> > > > will crack? Wrong question, not if, its when, if properly formed
> > > > as a question. :)
> > > >
> > > > > I hope to get myself an SSD to install as my home directory,
> > > > > which ought to speed things up for me, at least.
> > > >
> > > > Haveing done that to the boot drive in one of my milling
> > > > machines, I can testify that an old pentium powered Dell,
> > > > pulling from the sata SSD, is a good 10x faster than when it was
> > > > loading from spinning rust.  Amazing.
> > > >
> > > > > It is a problem of diminishing returns: you want to have those
> > > > > archives available somewhere, to find information that you
> > > > > saved, but now the size of those archives is making Kmail
> > > > > unusable. So I would say, keep pruning until it stops
> > > > > misbehaving. You will still have those old emails available,
> > > > > so long as they are removed to where Kmail won't look. They
> > > > > can always be retrieved if you really need them.
> > > > >
> > > > > Bill
> > > >
> > > > Take care Bill.
> > > >
> > > > Cheers, Gene Heskett
> > >
> > > Stupid question: Do you compact email folders? or does Kmail do
> > > that for you automatically every so periodically often? Mine is a
> > > little of both; if Kmail doesn't do it, then I compact them
> > > manually.
> > >
> > > Bill
> >
> > kmail does, I don't as a general rule. I'll see if I can do that, it
> > might just shrink the index files.
>
> No noticeable diff, whole maryann is still 4.8GB
>
> Thanks Bill.
Went to bed at 1:50, up to pee at 5:20, its back again.  Since sent-mail 
was the biggest, I went to the OldMail dir wih  mc and created a new 
old.sent.mail/cur there and moved the first half of the first year of 
sent-mail/cur to it.

stopped kmail
renamed/moved sent-mail/cur to old.sent.mail/gene
made a new sent-mail/cur
copied old.sent.mail/gene* to sent-mail/cur/
deleted old.sent.mail.gene

This should have given a newer, shorter directory with no blank spots 
from me moving stuff early in the directory out, leaving tons of empty 
space.

Restarted kmail, which took a long time to open its gui but hasn't 
crashed yet.  The regenerated sent-mail/cur now has a much shorter set 
of index files.  And it indexed it once and shut the indexing off, but 
must have had to think about it as it just started up again.  And I 
think what I'm seeing now for an ls -la Mail is from the cache, no files 
dates have changed in 6 minutes, but its still hammering away. And 
finally, after 14 minutes, updated the times of the debian folders 
index's.  At this rate, probably another 30 minutes to do the 2nd full 
scan.

I am reading each cores temp individually with gkrellm and its switching 
cores when the individual core hits 50C.

I think if this helps, I probably should do the cur rewrite to shorten 
the dir file as above on all the curs that have been divided into 
subdirs.

But, I don't think it helped. Its still hammering away, but hasn't 
updated an index file in over 20 minutes.  I may as well send this and 
go catch up on my sleep.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>