Gene Heskett wrote: > On Tuesday 15 October 2019 13:45:14 deloptes wrote: > >> Gene Heskett wrote: >> > Greetings all; >> >> Hi Gene, >> >> > Are you send those to /dev/null? I've sent a whole bunch of them. >> >> I do not know what you are talking about, but I recall you mentioned >> before you had issues with your mailbox/es. >> >> > These crashes always seem to be preceded by a couple days of a kmail >> > session burning up a core of 4 cores, bouncing to the next available >> > core, at 99 to 100% at 15 second or so intervals. >> >> AFAIR you are using local MailDir with tons of mail > > yes but its slowly disappearing. > BACKUP? mails can not be that big compared to nowdays drives capacity and prices >> > These seem to be related to kmail finding trash files in its >> > database, causing problems while re-indexing. I have two folders >> > which are subfolders with a given years messages manually sorted >> > into that years corpus. >> >> I recall experiencing similar with KDE3 like 12y ago. Unpleasant >> situation for sure! >> >> > When I relaunch kmail after one of these crashes, I am getting >> > advisories that so-and-so has an index problem, and its generally >> > the top level folder, and 2 or 3 of its subfolders that are named. >> > And the older folders are slowly being emptied, as in messages are >> > disappearing despite having no expiry set up. >> > >> > Can anything be done, or is there something I can check-uncheck >> > someplace? >> >> I don't know exactly, but I know what solved my problems was to setup >> imap server between the directory structure and the client. The server >> where the mails are is 4cores/32GB mem, however the server is not very >> fast - I am not sure why - I suspect the disks are not the fastest, >> but I do not experience any issue with mails. >> > I'm currently sucking, with fetchmail, from a dovecot database at my isp, > but the procmail direct deposit into the spam folder, which gets mv'd to > spam-hold for one day by the same script that has sa-learn spam being > done on that folder, moving what it has looked at to spam-hold. Thats > the only non-kmail accesses that I have programmed. Any other incoming > mail is deposited in /var/mail, and dbus sends kmail a check local > folder message for new mail by way of inotifywait seeing the closing of > the mail file and triggering the dbus message. The > locale /var/mail/mailfile generally has a lifetime in the non-zero > length state of milliseconds. > This is very complicated process. Are you sure kmail is working properly. Unfortunately to inspect kmail one should compile it with Debug enabled and install the additional package. So without closer inspection it is hard to say. I can only advise to simplify the process. This dovecot server for sure speaks imap to start with. So you could sync your own imap server and let the imap server be the interface to all your mails. >> Few years ago I integrated dbmail for a customer. It is extremely >> fast. Think about migrating those tons of mails. > > describe that please. > http://www.dbmail.org/dokuwiki/doku.php look at the project. AFAIR it can handle postgres or mysql. I used mysql and it was really impressive. In few words it is an IMAP server with database backend. Might be suitable, but also might be too much work (see below) compared to dovecot. I found dovecot more complex, but is widely used and with a lot of documentation and support around. >> I know it is not exactly an answer, but I can not think of any other. >> As for kmail I have not looked into it in detail, but it is complex - >> one has to be able to reproduce the problem to help you find a >> solution. >> >> Also are you sure nothing else is using your local Maildir? I know you >> have many scripts for this and that. > > procmail may, very occasionally place a particularly nasty bit of stuff > directly in the spam folder, but thats not more than a weekly > occurrence. If that often. I'll find that recipe and nuke it, just for > S&G. Or better yet, send that one to /dev/null. At least I'll have a > log entry. Right now, I think I'll go over the /home/gene/Mail dir and > nuke every sorted index just for S&G as thats nowhere near universally > present. Then I'll exit kmail as the uptime is 29+days, and restart it. > I don't know Gene. I have the suspicion that there are two things working on your maildir corrupting the files and I think you overcomplicated the things. There is a program called imapsync for example. If I were you and I wanted to save mails locally, I would use imapsync with either dovecot or dbmail and use the imap interface to read whatever mails I want with kmail - tends to work pretty well for me for over 10y now. I use two imap servers for mail with kmail - the local one and the public one. There might be around 10000msg there at the moment - most of them on the public server. I do not sync them locally, but I was playing with this idea via imapsync. I used imapsync in one company few years ago and it is very robust application. Probably the easiest way is to let dovecot handle your Maildir (don't forget to relocate it outside of your ~/ and let kmail forget it). I think this is what I finally ended up doing may be 10y ago. Since then 0 problems. I hope it helps regards