Commit bb0fd7c
feat: message previews
Part of #7367
- Remove partial downloads (remove creation of the stub messages) (#7373)
- Remove "Download maximum available until" and remove stock string `DC_STR_DOWNLOAD_AVAILABILITY` (#7369)
- Send pre-message on messages with large attachments (#7410)
- Pre messages can now get read receipts (#7433)
progress / what's to do:
- [x] send pre-message
- [x] The message's state must be set to MessageState::OutDelivered only
after both messages are sent. If a read receipt is received, the message
can be OutMdnRcvd or OutPending; let's just do whatever is easiest for
now. Take care not to revert from OutMdnReceived to OutDelivered if we
first receive a read receipt and then deliver the full message.
- this is already the case:
- `OutDelivered` is set when a message is sent out and has no remaining
send jobs in the smtp table for this message id - so already works since
full message and pre message have same msgId in that table.
- `OutMdnRcvd` is a "virtual" state
(#7367 (comment)),
so going back to `OutDelivered` can't happen anymore
- [x] delimit `ChatFullMessageId` with `<` and `>` like the other
message ids
- [x] add tests
- [x] test that pre message is sent for attachment larger than X
- test that correct headers are present on both messages
- also test that Autocrypt-gossip and selfavatar should never go into
full-messages
- [x] test that no pre message is sent for attachment smaller than X
- no "is full message" header should be present
- [x] test that pre message is not send for large webxdc update or large
text
- [x] fix test `receive_imf::receive_imf_tests::test_dont_reverify_by_self_on_outgoing_msg`
feat: receive pre-messages and adapt download on demand
fix python lint errors
receive pre-mesages, start with changes to imap loop.
refactor: move download code from `scheduler.rs` to `download.rs`, also
move `get_msg_id_by_rfc724_mid` to `MsgId::get_by_rfc724_mid`
`MAX_FETCH_MSG_SIZE` is no longer unused
Parse if it is a pre-message or full-message
start with receiving logic
get rid of `MsgId::get_by_rfc724_mid` because it was a duplicate of
`message::rfc724_mid_exists`
docs: add hint to `MimeMessage::from_bytes` stating that it has
side-effects.
receiving full message
send and receive `attachment_size` and set viewtype to text in
pre_message
metadata as struct in pre-message in header. And fill params that we can
already fill from the metadata. Also add a new api to check what
viewtype the message will have once downloaded.
api: jsonrpc: add `full_message_view_type` to `Message` and
`MessageInfo`
make PreMsgMetadata.to_header_value not consume self/PreMsgMetadata
add api to merge params
on download full message: merge new params into old params and remove
full-message metadata params
move tests to `src/tests/pre_messages.rs`
dynamically allocate test attachment bytes
fix detection of pre-messages. (it looked for the ChatFullMessageId
header in the unencrypted headers before)
fix setting dl state to avaiable on pre-messages
fix: save pre message with rfc724_mid of full message als disable
replacement for full messages
add some receiving tests and update test todo for premessage metadata
test: process full message before pre-message
test receive normal message
some serialization tests for PreMsgMetadata
remove outdated todo comment
test that pre-message contains message text
PreMsgMetadata: test_build_from_file_msg and test_build_from_file_msg
test: test_receive_pre_message_image
Test receiving the full message after receiving an edit after receiving
the pre-message
test_reaction_on_pre_message
test_full_download_after_trashed
test_webxdc_update_for_not_downloaded_instance
simplify fake webxdc generation in
test_webxdc_update_for_not_downloaded_instance
test_markseen_pre_msg
test_pre_msg_can_start_chat and test_full_msg_can_start_chat
test_download_later_keeps_message_order
test_chatlist_event_on_full_msg_download
fix download not working
log splitting into pre-message
add pre-message info to text when loading from db. this can be disabled
with config key `hide_pre_message_metadata_text` if ui wants to display
it in a prettier way.
update `download_limit` documentation
more logging: log size of pre and post messages
rename full message to Post-Message
split up the pre-message tests into multiple files
dedup test code by extracting code to create test messages into util
methods
remove post_message_view_type from api, now it is only used internally
for tests
remove `hide_pre_message_metadata_text` config option, as there
currently is no way to get the full message viewtype anymore
Update src/download.rs
resolve comment
use `parse_message_id` instead of removing `<>`parenthesis it manually
fix available_post_msgs gets no entries
handle forwarding and add a test for it.
convert comment to log warning event on unexpected download failure
add doc comment to `simple_imap_loop`
more logging
handle saving pre-message to self messages and test.
refactor: Simplify the logic for downloading a bit
Immediately fully download all messages that do not have the
`Chat-Is-Post-Message` header. This way, we simplify the logic for when
which messages are downloaded, there are no differences at all anymore
between 'background_fetch' and 'normal fetch'.
Also, we prevent message reordering when reveiving a message from a
legacy client.
Messages larger than 1MB without `Chat-Is-Post-Message` should be rare,
so, we do not expect this to really worsen things.
Update deltachat-ffi/deltachat.h
less .clone()
clippy fixes
rustfmt
restore test_download_on_demand
use anyhow
remove TODO
remove PartialDownloadMsgBody
Add DC_STR_CONTACT stock string
remove stock string TODOs
remove optional todo
feat: Don't put text into post-message
This way, there will be no duplicate text if a chat partner is using an older DC
version.
If the chat partner is using a new DC version, then this will have no
effect.
fix: Make pre-messages db migration backwards-compatible
See this comment that described the problem: https://github.com/chatmail/core/pull/7371/files#r2612182387
> The migration removes the `msg_id` column from the `download` table, so that, if you import the backup into an older DC version, you get the error `Failed inbox fetch_idle: Failed to download messages: no such column: msg_id in SELECT msg_id FROM download at offset 7: Error code 1: SQL error or missing database.`
>
> So, we need:
> - either to make sure that things somewhat work if the user downgrades
> - or to again increase `backup_version`, and then again coordinate that releases are done on all platforms simultaneously (which I would like to avoid).
This PR here implements the first solution: Re-add `download.msg_id`, so that an older core can handle the database scheme.
Co-authored-by: Hocuri <hocuri@gmx.de>1 parent 5925f72 commit bb0fd7c
File tree
41 files changed
+2411
-1446
lines changed- deltachat-ffi
- deltachat-jsonrpc/src/api/types
- deltachat-rpc-client/tests
- python/tests
- src
- calls
- chat
- context
- download
- imap
- message
- mimefactory
- mimeparser
- receive_imf
- sql
- stock_str
- tests
- pre_messages
- webxdc
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
41 files changed
+2411
-1446
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
488 | 488 | | |
489 | 489 | | |
490 | 490 | | |
491 | | - | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
492 | 494 | | |
493 | | - | |
494 | | - | |
495 | | - | |
| 495 | + | |
496 | 496 | | |
497 | 497 | | |
498 | 498 | | |
| |||
4310 | 4310 | | |
4311 | 4311 | | |
4312 | 4312 | | |
4313 | | - | |
| 4313 | + | |
| 4314 | + | |
4314 | 4315 | | |
4315 | 4316 | | |
4316 | 4317 | | |
| |||
7263 | 7264 | | |
7264 | 7265 | | |
7265 | 7266 | | |
7266 | | - | |
7267 | | - | |
7268 | | - | |
7269 | | - | |
7270 | | - | |
7271 | | - | |
7272 | | - | |
| 7267 | + | |
7273 | 7268 | | |
7274 | 7269 | | |
7275 | | - | |
7276 | | - | |
7277 | | - | |
7278 | | - | |
7279 | | - | |
7280 | | - | |
7281 | | - | |
7282 | 7270 | | |
7283 | 7271 | | |
7284 | 7272 | | |
| |||
7776 | 7764 | | |
7777 | 7765 | | |
7778 | 7766 | | |
| 7767 | + | |
| 7768 | + | |
| 7769 | + | |
| 7770 | + | |
| 7771 | + | |
7779 | 7772 | | |
7780 | 7773 | | |
7781 | 7774 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
95 | 98 | | |
96 | 99 | | |
97 | 100 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | | - | |
5 | 3 | | |
6 | 4 | | |
7 | 5 | | |
| |||
129 | 127 | | |
130 | 128 | | |
131 | 129 | | |
132 | | - | |
| 130 | + | |
133 | 131 | | |
134 | 132 | | |
135 | 133 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | 8 | | |
10 | 9 | | |
11 | 10 | | |
12 | 11 | | |
13 | | - | |
14 | | - | |
| 12 | + | |
| 13 | + | |
15 | 14 | | |
16 | 15 | | |
17 | 16 | | |
| |||
333 | 332 | | |
334 | 333 | | |
335 | 334 | | |
336 | | - | |
| 335 | + | |
337 | 336 | | |
338 | 337 | | |
339 | 338 | | |
| |||
342 | 341 | | |
343 | 342 | | |
344 | 343 | | |
345 | | - | |
| 344 | + | |
346 | 345 | | |
347 | 346 | | |
348 | 347 | | |
349 | 348 | | |
350 | | - | |
| 349 | + | |
351 | 350 | | |
352 | 351 | | |
353 | 352 | | |
354 | 353 | | |
355 | | - | |
356 | 354 | | |
357 | 355 | | |
358 | 356 | | |
| |||
588 | 586 | | |
589 | 587 | | |
590 | 588 | | |
591 | | - | |
592 | | - | |
593 | | - | |
594 | | - | |
595 | | - | |
596 | | - | |
597 | | - | |
598 | | - | |
599 | | - | |
600 | | - | |
601 | | - | |
602 | | - | |
603 | | - | |
604 | | - | |
605 | | - | |
606 | | - | |
607 | | - | |
608 | | - | |
609 | | - | |
610 | | - | |
611 | | - | |
612 | | - | |
613 | | - | |
614 | | - | |
615 | | - | |
616 | | - | |
617 | | - | |
618 | | - | |
619 | | - | |
620 | | - | |
621 | | - | |
622 | | - | |
623 | | - | |
624 | | - | |
625 | | - | |
626 | | - | |
627 | | - | |
628 | | - | |
629 | | - | |
630 | | - | |
631 | | - | |
632 | | - | |
633 | | - | |
634 | | - | |
635 | | - | |
636 | | - | |
637 | | - | |
638 | | - | |
639 | | - | |
640 | | - | |
641 | | - | |
642 | | - | |
643 | | - | |
644 | | - | |
645 | | - | |
646 | | - | |
647 | | - | |
648 | | - | |
649 | | - | |
650 | | - | |
651 | | - | |
652 | | - | |
653 | | - | |
654 | | - | |
655 | | - | |
656 | | - | |
657 | | - | |
658 | | - | |
659 | | - | |
660 | | - | |
661 | | - | |
662 | | - | |
663 | | - | |
664 | | - | |
665 | | - | |
666 | | - | |
667 | | - | |
668 | | - | |
669 | | - | |
670 | | - | |
671 | | - | |
672 | | - | |
673 | | - | |
674 | | - | |
675 | | - | |
676 | | - | |
677 | | - | |
678 | | - | |
679 | 589 | | |
680 | 590 | | |
681 | 591 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
5 | 4 | | |
6 | 5 | | |
7 | 6 | | |
| |||
222 | 221 | | |
223 | 222 | | |
224 | 223 | | |
225 | | - | |
226 | | - | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
238 | | - | |
239 | | - | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
255 | | - | |
256 | | - | |
257 | 224 | | |
258 | 225 | | |
259 | 226 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
610 | 610 | | |
611 | 611 | | |
612 | 612 | | |
613 | | - | |
614 | | - | |
615 | | - | |
616 | | - | |
617 | | - | |
618 | | - | |
619 | | - | |
620 | | - | |
621 | | - | |
622 | | - | |
623 | | - | |
624 | | - | |
625 | | - | |
626 | | - | |
627 | | - | |
628 | | - | |
629 | | - | |
630 | | - | |
631 | | - | |
632 | | - | |
633 | | - | |
634 | | - | |
635 | | - | |
636 | | - | |
637 | | - | |
638 | | - | |
639 | | - | |
640 | | - | |
641 | | - | |
642 | | - | |
643 | | - | |
644 | | - | |
645 | | - | |
646 | | - | |
647 | | - | |
648 | | - | |
649 | | - | |
650 | | - | |
651 | | - | |
652 | | - | |
653 | | - | |
654 | | - | |
655 | | - | |
656 | | - | |
657 | | - | |
658 | | - | |
659 | | - | |
660 | | - | |
661 | | - | |
662 | | - | |
663 | | - | |
664 | | - | |
665 | | - | |
666 | | - | |
667 | | - | |
668 | | - | |
669 | | - | |
670 | | - | |
671 | | - | |
672 | | - | |
673 | | - | |
674 | | - | |
0 commit comments