Arnt Gulbrandsen [Thu, 20 Nov 2008 14:51:19 +0000 (15:51 +0100)]
avoid pgsql warning
Arnt Gulbrandsen [Thu, 20 Nov 2008 12:44:50 +0000 (13:44 +0100)]
IMAP EXAMINE was too broken for squirrelmail. fix.
Abhijit Menon-Sen [Thu, 20 Nov 2008 11:26:46 +0000 (16:56 +0530)]
Fix typo.
Arnt Gulbrandsen [Wed, 19 Nov 2008 13:19:43 +0000 (14:19 +0100)]
plug leak
Abhijit Menon-Sen [Wed, 19 Nov 2008 13:04:44 +0000 (18:34 +0530)]
Integrate another line that got missed from
Arnt Gulbrandsen [Wed, 19 Nov 2008 11:51:02 +0000 (12:51 +0100)]
dot an i.
the session is too much of an imap thing. pop needs to bend over and
pretend to be imap.
Arnt Gulbrandsen [Wed, 19 Nov 2008 11:41:30 +0000 (12:41 +0100)]
superclass virtual changed signature; follow the change in subclass
Abhijit Menon-Sen [Wed, 19 Nov 2008 11:35:28 +0000 (17:05 +0530)]
Internaldate is also trivia.
(One line missed due to a bad merge earlier.)
Abhijit Menon-Sen [Wed, 19 Nov 2008 11:30:28 +0000 (17:00 +0530)]
Remove some debug logging.
Abhijit Menon-Sen [Wed, 19 Nov 2008 11:28:24 +0000 (16:58 +0530)]
Revert "udoc fixes."
This reverts part of commit
6991eb2b7bf49592d60e653a8fc9d657686941aa,
which depends on a patch that was not integrated into 3.0.x.
Abhijit Menon-Sen [Wed, 19 Nov 2008 11:25:44 +0000 (16:55 +0530)]
Revert "code changed; doc needs to change"
This reverts commit
b3423fe5a118225e948c1d2f88bfc07817fa92fe, which was
dependent on another patch that was not integrated.
Arnt Gulbrandsen [Wed, 19 Nov 2008 11:07:15 +0000 (12:07 +0100)]
if we need to send the query, then we need to ask for all the results.
(the code below assumes that the results are complete.)
Arnt Gulbrandsen [Wed, 19 Nov 2008 10:14:46 +0000 (11:14 +0100)]
duplicating state is bad. clean up.
Arnt Gulbrandsen [Wed, 19 Nov 2008 10:12:13 +0000 (11:12 +0100)]
better logging in case a large sparse UID set is used
Arnt Gulbrandsen [Thu, 6 Nov 2008 11:15:39 +0000 (12:15 +0100)]
simpler (faster?) IMAP COPY.
a side effect is that a concurrent expunge could break the COPYUID
response, but not any more.
(Was meant for 3.1, but it's convenient to integrate it here because of
the changes to Mailbox to make it use subtransactions. -- AMS)
Arnt Gulbrandsen [Tue, 18 Nov 2008 15:33:50 +0000 (16:33 +0100)]
try to free up garbage sooner if we're over 2*limit
if limit is 0.... hm?
Arnt Gulbrandsen [Tue, 18 Nov 2008 15:27:37 +0000 (16:27 +0100)]
use mmap() to provide memory if MAP_ANON is available, else malloc.
this might relieve memory stress, by making us able to drop RAM back to
the OS.
Arnt Gulbrandsen [Tue, 18 Nov 2008 14:38:40 +0000 (15:38 +0100)]
notify the owner whenever the fetcher has achieved something.
the owner may be able to use the results, even if other results are
missing still.
Arnt Gulbrandsen [Tue, 18 Nov 2008 14:33:57 +0000 (15:33 +0100)]
try to use really small batches with pg81, to avoid seqscans on header_fields
I know we should be raising some statistics target, but even we aren't
doing that, so...
Arnt Gulbrandsen [Tue, 18 Nov 2008 14:28:11 +0000 (15:28 +0100)]
emit fetch responses sooner (don't wait for the end of the commmand)
helps fight memory blowup
Arnt Gulbrandsen [Tue, 18 Nov 2008 09:53:29 +0000 (10:53 +0100)]
avoid those scary noops.
we'd enqueue the deliveries change n times, even after commit, and log
'delivered' n times too.
Arnt Gulbrandsen [Tue, 18 Nov 2008 09:23:08 +0000 (10:23 +0100)]
rollback in case two processes service the spool at teh same time
Arnt Gulbrandsen [Tue, 18 Nov 2008 09:13:50 +0000 (10:13 +0100)]
the ocd manpage is gone. compile.
Abhijit Menon-Sen [Tue, 18 Nov 2008 03:32:08 +0000 (09:02 +0530)]
Get rid of ocd manpage.
Thanks for pointing it out, Germán.
Arnt Gulbrandsen [Fri, 14 Nov 2008 11:41:38 +0000 (12:41 +0100)]
make store finish its work provided that the session is active when it starts.
earlier, the store would stop working if the imap session closed while
store was working. which shouldn't happen but sometimes clients are
naughty. now, Store picks up a session when execute() is first called, and
uses that until it's done.
it would also be possible to have Store abort cleanly. I think I prefer to
finish.
Arnt Gulbrandsen [Fri, 14 Nov 2008 11:36:07 +0000 (12:36 +0100)]
don't notify the parent's owner when a subtransaction concludes. not our job.
Arnt Gulbrandsen [Fri, 14 Nov 2008 11:35:49 +0000 (12:35 +0100)]
more documentation, more comments, particularly for subtransactions
Arnt Gulbrandsen [Fri, 14 Nov 2008 11:33:43 +0000 (12:33 +0100)]
don't rely on a subtransaction to notify its parent's owner when it concludes.
simplies the code nicely, overall.
3.1 stuff.
Arnt Gulbrandsen [Thu, 13 Nov 2008 15:29:30 +0000 (16:29 +0100)]
unfucked version of previous commit.
don't send '' as an integer if the array is empty
Arnt Gulbrandsen [Thu, 13 Nov 2008 15:20:32 +0000 (16:20 +0100)]
change lock order so that if we lock both mm and mailboxes, we lock mm LAST.
a non-peek fetch may now lock the mailbox, if it retrieves both the modseq
and needs to set \seen. unfortunate, but if we didn't do this, then the
store could lock the mailbox after fetch locks mm, and the sub-fetch used
to report blah could deadlock against this or another fetch. (even against
itself because the store wasn't even a subtransaction until now.)
store is a subtransaction so that a modseq non-peek fetch won't deadlock
against itself via store and the flag update.
store and expunge lock mailboxes before mailbox_messages.
Arnt Gulbrandsen [Thu, 13 Nov 2008 15:18:48 +0000 (16:18 +0100)]
code changed; doc needs to change
Arnt Gulbrandsen [Thu, 13 Nov 2008 12:09:20 +0000 (13:09 +0100)]
unbreak queries that use empty arrays.
(1 is not the only number that's smaller than 2.)
Arnt Gulbrandsen [Thu, 13 Nov 2008 11:43:44 +0000 (12:43 +0100)]
exit dead code
Arnt Gulbrandsen [Thu, 13 Nov 2008 11:38:27 +0000 (12:38 +0100)]
sometimes the injector is called with zero messages. log that gracefully
and consider everything done.
Arnt Gulbrandsen [Thu, 13 Nov 2008 11:29:45 +0000 (12:29 +0100)]
handle injection of messages with zero bodyparts
Arnt Gulbrandsen [Thu, 13 Nov 2008 10:59:25 +0000 (11:59 +0100)]
all transactions must have owners.
the rule has to apply to subtransactions, because we assume it to be true
and I don't care to look for the code that assumes so. easier to make the
rule hole.
Abhijit Menon-Sen [Thu, 13 Nov 2008 10:10:34 +0000 (15:40 +0530)]
Update VERSION.
Arnt Gulbrandsen [Wed, 12 Nov 2008 15:28:08 +0000 (16:28 +0100)]
make memory-limit apply to archiveopteryx only
tlsproxy and logd collect memory down to zero anyway
Arnt Gulbrandsen [Wed, 12 Nov 2008 15:16:02 +0000 (16:16 +0100)]
debug logging to catch possible problems wrt. large unflushed data
Arnt Gulbrandsen [Wed, 12 Nov 2008 14:42:31 +0000 (15:42 +0100)]
collect garbage only if discarding the cache makes sense.
garbage allocation is slow if the result set is large. so let's just not
collect garbage unless we'll the cache.
Arnt Gulbrandsen [Wed, 12 Nov 2008 08:01:46 +0000 (09:01 +0100)]
build on linux/ppc
tested by aox@visteya.net; must ask for retesting
Arnt Gulbrandsen [Tue, 11 Nov 2008 14:11:08 +0000 (15:11 +0100)]
report on what's done. number of messages, mailbox.
Arnt Gulbrandsen [Tue, 11 Nov 2008 13:57:12 +0000 (14:57 +0100)]
aox undelete misused generate_series. use a sequence instead.
this works.
Arnt Gulbrandsen [Tue, 11 Nov 2008 13:42:12 +0000 (14:42 +0100)]
log the die()
suse the Log ID that's in scope when die() is called; that'll probably be
the right one.
doesn't log die( Memory ) since that's often called within Allocator, so
we may not be able to allocate any memory for logging.
Arnt Gulbrandsen [Tue, 11 Nov 2008 13:31:31 +0000 (14:31 +0100)]
messageset doesn't benefit from that sorting any more, so don't bother
Arnt Gulbrandsen [Tue, 11 Nov 2008 11:53:51 +0000 (12:53 +0100)]
debug logging away again
Arnt Gulbrandsen [Tue, 11 Nov 2008 11:51:53 +0000 (12:51 +0100)]
if we log a client out, we tell it NO, but the commands it has sent aren't
all that BAD.
Arnt Gulbrandsen [Tue, 11 Nov 2008 11:43:26 +0000 (12:43 +0100)]
better error messages in case an IMAP command isn't permitted due to the
IMAP state.
if the old message could confuse us, it could confuse anyone.
Arnt Gulbrandsen [Tue, 11 Nov 2008 11:13:48 +0000 (12:13 +0100)]
make Transaction::execute() fail all queries if it can't.
the most common case would be if the parent has already been committed or
rolled back.
Abhijit Menon-Sen [Tue, 11 Nov 2008 11:01:56 +0000 (16:31 +0530)]
udoc fix.
Arnt Gulbrandsen [Tue, 11 Nov 2008 10:48:12 +0000 (11:48 +0100)]
ask the db whether we're blocked instead of maintaining duplicate state
make rollback() work for subtransactions (the release savepoint wasn't sent)
Arnt Gulbrandsen [Mon, 10 Nov 2008 14:39:49 +0000 (15:39 +0100)]
avoid casting pointers to int, it has bad karma
Arnt Gulbrandsen [Mon, 10 Nov 2008 14:13:23 +0000 (15:13 +0100)]
the merge didn't know that I had just removed idCache
Abhijit Menon-Sen [Mon, 10 Nov 2008 13:36:54 +0000 (19:06 +0530)]
Another idate.
But is this the only problem?
Arnt Gulbrandsen [Mon, 10 Nov 2008 13:17:49 +0000 (14:17 +0100)]
make PreparedStatement not be freed.
as of this change, it's no longer necessary to addEternal every single
PreparedStatement.
Arnt Gulbrandsen [Mon, 10 Nov 2008 13:10:54 +0000 (14:10 +0100)]
drop the addresscache map from id. it's not used.
also simplify some code; there isn't any point to avoiding those pointers
since we'll drop the cache when we drop those buffers.
Arnt Gulbrandsen [Mon, 10 Nov 2008 11:54:41 +0000 (12:54 +0100)]
get rid of the address cache when we run up against the memory limit.
this makes addresses behave like messages.
Abhijit Menon-Sen [Sun, 9 Nov 2008 02:50:49 +0000 (08:20 +0530)]
Another arrays vs. Postgres 8.1 hack.
If we use =any($x) where $x is bound to a single-element array, we
rewrite both query and parameter so that we can use =$x instead.
QV::position() changes type to avoid comparison warnings.
Untested.
Arnt Gulbrandsen [Sat, 8 Nov 2008 23:09:22 +0000 (00:09 +0100)]
I wish it weren't true
Arnt Gulbrandsen [Sat, 8 Nov 2008 10:55:34 +0000 (11:55 +0100)]
support duplicate message ids in one mailbox
(a message copied twice into one mailbox, both copies fetched with one
fetch)
Arnt Gulbrandsen [Fri, 7 Nov 2008 21:10:22 +0000 (22:10 +0100)]
was broken for really large sets (e.g. value=100000, i=20000)
Arnt Gulbrandsen [Fri, 7 Nov 2008 20:43:00 +0000 (21:43 +0100)]
make forwarding submitted mail to the smarthost work
Arnt Gulbrandsen [Fri, 7 Nov 2008 20:38:35 +0000 (21:38 +0100)]
compile$#@$!@#
Arnt Gulbrandsen [Fri, 7 Nov 2008 20:28:02 +0000 (21:28 +0100)]
injecting only into deliveries didn't work. fix.
DA still doesn't work, not sure why, will find out.
Arnt Gulbrandsen [Fri, 7 Nov 2008 19:52:39 +0000 (20:52 +0100)]
change the slashes in the transaction ID to dashes for the copy filename
Arnt Gulbrandsen [Fri, 7 Nov 2008 19:46:39 +0000 (20:46 +0100)]
accept LIST "/" ""
Arnt Gulbrandsen [Fri, 7 Nov 2008 14:53:44 +0000 (15:53 +0100)]
send OK [CLOSED] only when changing sessions, not when closing.
imaptest complained.
Arnt Gulbrandsen [Fri, 7 Nov 2008 14:37:27 +0000 (15:37 +0100)]
don't shut down the server if the mailbox reader fails during an update
the update can fail if there's a lock problem, e.g. if store attempts to
lock a message and there's a deadlock. there shouldn't be deadlocks, but
if there are, we still don't want to shutdown.
Arnt Gulbrandsen [Fri, 7 Nov 2008 14:30:27 +0000 (15:30 +0100)]
don't enter an infinite loop in select
Arnt Gulbrandsen [Fri, 7 Nov 2008 14:25:36 +0000 (15:25 +0100)]
if we've created one EXPUNGE response, that's enough. avoid repeating.
Arnt Gulbrandsen [Fri, 7 Nov 2008 13:57:01 +0000 (14:57 +0100)]
if a fetch uses a transaciton, it has to commit it, even if the fetch
turns out to be a no-op.
Arnt Gulbrandsen [Fri, 7 Nov 2008 13:26:39 +0000 (14:26 +0100)]
unfuck expunge
Arnt Gulbrandsen [Fri, 7 Nov 2008 12:44:57 +0000 (13:44 +0100)]
sublogs to match subtransactions
Arnt Gulbrandsen [Fri, 7 Nov 2008 12:23:20 +0000 (13:23 +0100)]
commit ASAP, not later
Arnt Gulbrandsen [Fri, 7 Nov 2008 11:28:27 +0000 (12:28 +0100)]
if we suffer a timeout, abort _all_ parent transactions as well as the
active subtransaction
Arnt Gulbrandsen [Fri, 7 Nov 2008 11:08:59 +0000 (12:08 +0100)]
more logging, just for debugging.
and a cleanup or two.
Arnt Gulbrandsen [Fri, 7 Nov 2008 11:05:51 +0000 (12:05 +0100)]
one fewer Flag user
Arnt Gulbrandsen [Fri, 7 Nov 2008 10:02:33 +0000 (11:02 +0100)]
decode each message in one go, so that we don't confuse the cache
two fetchers could independently of each other see that a message didn't
have e.g. address fields, decide to fetch them, fetch them, and call
setAddressesFetched().
now, the fetcher checks, decodes and records atomically. if another
fetcher has just stored the information, the fetcher will discard the
rows.
Arnt Gulbrandsen [Fri, 7 Nov 2008 09:40:58 +0000 (10:40 +0100)]
if the value was smaller than smallest(), bad things could happen. fix.
Arnt Gulbrandsen [Fri, 7 Nov 2008 07:51:26 +0000 (08:51 +0100)]
if a UID has been expunged, don't wait for it
Abhijit Menon-Sen [Thu, 6 Nov 2008 22:44:56 +0000 (04:14 +0530)]
udoc fixes.
Arnt Gulbrandsen [Thu, 6 Nov 2008 21:33:16 +0000 (22:33 +0100)]
reveal exactly when we send the tagged OK
Arnt Gulbrandsen [Thu, 6 Nov 2008 21:32:31 +0000 (22:32 +0100)]
Transaction::restart() can work for non-subtransactions, so let it
more logging; should be helpful in case a subtransaction calls begin too late
Arnt Gulbrandsen [Thu, 6 Nov 2008 21:30:51 +0000 (22:30 +0100)]
even fetch may have to lock messages, and if so it should use the same
sequence as expunge and store.
but let's not lock very much in fetch. we don't lock just for flags. if a
condstore client is there, we lock, otherwise not.
Arnt Gulbrandsen [Thu, 6 Nov 2008 21:19:24 +0000 (22:19 +0100)]
don't segfault even if there is a database deadlock
Arnt Gulbrandsen [Thu, 6 Nov 2008 21:18:49 +0000 (22:18 +0100)]
segfault if the client closes during store .silent
Arnt Gulbrandsen [Thu, 6 Nov 2008 20:55:05 +0000 (21:55 +0100)]
expunge too needs to use the same order
Arnt Gulbrandsen [Thu, 6 Nov 2008 20:54:26 +0000 (21:54 +0100)]
prevent deadlock; lock in a defined order
Arnt Gulbrandsen [Thu, 6 Nov 2008 20:25:43 +0000 (21:25 +0100)]
make session abort work even the reason is outside this process
the mailboxreader will learn that the mailbox disappeared, tell the
mailbox, and the mailbox aborts its sessions.
Arnt Gulbrandsen [Thu, 6 Nov 2008 17:08:18 +0000 (18:08 +0100)]
we cannot optimise away a flag search just because Flag doesn't know the
name. Flag is not omniscient.
Arnt Gulbrandsen [Thu, 6 Nov 2008 16:55:47 +0000 (17:55 +0100)]
join in flag_names instead of hoping for Flag to be correct.
Good bye, Flag.
Arnt Gulbrandsen [Thu, 6 Nov 2008 16:44:14 +0000 (17:44 +0100)]
create has to wait until it's done its work
Arnt Gulbrandsen [Thu, 6 Nov 2008 15:44:32 +0000 (16:44 +0100)]
a fetch that both retrieved the modseq and caused to increase would break
it would retrieve the modseq, then run store, then publish the modseq it
retrieved, but the store might have increased it earlier, so breakage
happened.
this causes a performance loss for condstore-using clients. oh no.
Arnt Gulbrandsen [Thu, 6 Nov 2008 15:29:22 +0000 (16:29 +0100)]
make plain 'fetch modseq' work when some but not all of the messages are
available in the message cache.
Arnt Gulbrandsen [Thu, 6 Nov 2008 15:23:10 +0000 (16:23 +0100)]
now that imapurlfetcher uses the message cache, it can finish during the
first call to execute(), which append() didn't expect. fix.
Arnt Gulbrandsen [Thu, 6 Nov 2008 14:55:04 +0000 (15:55 +0100)]
Injector::announce was a no-op
Arnt Gulbrandsen [Thu, 6 Nov 2008 14:41:55 +0000 (15:41 +0100)]
an infinite loop. begone!
Arnt Gulbrandsen [Thu, 6 Nov 2008 14:19:19 +0000 (15:19 +0100)]
use subtransactions to update the mailbox tree, sessions, and flags.
when we update the databases, we generally need to do some querying and
reporting. this change makes those go into the updating transaction, so
that the updates happen at completely predictable times.
seems to work, although the number of savepoints may be too high. some
state machine may be bad.
there is a problem - if e.g. the injector's transaction fails to commit,
then we've already updated sessions. the only reasonable way out of that
is to abort those sessions.
Arnt Gulbrandsen [Thu, 6 Nov 2008 14:18:31 +0000 (15:18 +0100)]
injected() used the old state machine numbers. update.
also publish injector errors to the sieve user.
Arnt Gulbrandsen [Thu, 6 Nov 2008 12:31:13 +0000 (13:31 +0100)]
coarse mtime strikes again