Commit Graph

57 Commits

Author SHA1 Message Date
Daniel Gasienica
1dd87ad197 Format all source code using Prettier 2018-04-30 16:53:34 -04:00
Daniel Gasienica
d00fb560b0 Avoid exposing AttachmentTS 2018-04-27 16:31:43 -04:00
Daniel Gasienica
f36f206a01 Use IndexablePresence for hasFileAttachments and hasVisualMediaAttachments
Reduces index size, makes it easier to debug using IndexedDB inspector, and
hopefully improves lookup performance.
2018-04-25 15:25:12 -04:00
Daniel Gasienica
8e3c38d5fe Fix formatting 2018-04-25 15:25:12 -04:00
Daniel Gasienica
146178f977 Update schema documentation 2018-04-25 15:24:51 -04:00
Daniel Gasienica
9d84b2f420 Index messages with attachments using a boolean
When indexing message attachment metadata using numeric indexes such as:

```javascript
{
  conversationId: '+12223334455',
  received_at: 123,
  attachments: […],
  numAttachments: 2,
},
{
  conversationId: '+12223334455',
  received_at: 456,
  attachments: [],
  numAttachments: 0,
}
{
  conversationId: '+12223334455',
  received_at: 789,
  attachments: [],
  numAttachments: 1,
}
```

It creates an index as follows:

```
[conversationId, received_at, numAttachments]
['+12223334455', 123, 2]
['+12223334455', 456, 0]
['+12223334455', 789, 1]
```

This means a query such as…

```
lowerBound: ['+12223334455', 0,                1               ]
upperBound: ['+12223334455', Number.MAX_VALUE, Number.MAX_VALUE]
```

…will return all three original entries because they span the `received_at`
from `0` through `Number.MAX_VALUE`. One workaround is to index booleans using
`1 | undefined` where `1` is included in the index and `undefined` is not, but
that way we lose the ability to query for the `false` value. Instead, we flip
adjust the index to `[conversationId, hasAttachments, received_at]` and can
then query messages with attachments using

```
[conversationId, 1 /* hasAttachments */, 0                /* received_at */]
[conversationId, 1 /* hasAttachments */, Number.MAX_VALUE /* received_at */]
```
2018-04-25 15:24:51 -04:00
Daniel Gasienica
d14761087a Add schema version 4: Attachment metadata 2018-04-25 15:24:50 -04:00
Daniel Gasienica
df2e6e7864 Port MIME module to TypeScript 2018-04-25 15:24:50 -04:00
Daniel Gasienica
6a63e427c8 Use is instead of Lodash is* 2018-04-25 15:24:50 -04:00
Scott Nonnenberg
bf81c3db63 Document Message v4, remove data-free thumbnails 2018-04-23 16:43:25 -07:00
Scott Nonnenberg
c02860af5c Responding to pull request review feedback
- messages.getQuoteObjectUrl: early return
- backup.js: explaining variables for long if statement
- types/messages.js: Log if thumbnail has neither data nor path
- sendmessage.js:
  - remove extraneous logging
  - fix indentation
  - upload attachments and thumbnails in parallel
- preload: don't load fs for tests, just fse
- _conversation.scss: split two selectors into two lines, 0px -> 0
- backup_test.js: use fse.existsSync and comment twoSlashes regex
- network_tests_view_test.js: Comment duplicate assignment to window.getSocketStatus
2018-04-23 15:36:47 -07:00
Scott Nonnenberg
a7d44d3344 Backup and end-to-end test! 2018-04-20 15:24:04 -07:00
Scott Nonnenberg
f1ff040842 Finish type definition for _mapQuotedAttachments 2018-04-16 12:16:34 -07:00
Scott Nonnenberg
9ad55c803f Fix handling attachment thumbnails using thumbnail key 2018-04-13 18:10:51 -07:00
Scott Nonnenberg
054d3887a1 Quotes: The full pipeline into the database
1. MessageReceiver always pulls down thumbnails included in quotes
2. Message.upgradeSchema has a new schema that puts all thumbnails on
   disk just like happens with full attachments.
3. handleDataMessage pipes quote from dataMessage into the final message
   destined for the database
2018-04-13 18:05:45 -07:00
Scott Nonnenberg
21bf02c94d Fixed examples in Quote.md, rough Android visuals 2018-04-13 18:05:44 -07:00
Daniel Gasienica
24f4ad53bc Use single quotes for identifiers 2018-04-11 15:54:32 -04:00
Daniel Gasienica
195e9b777c Rename willHaveAttachmentsSavedOnFileSystemDuringUpgrade
New name: `willAttachmentsGoToFileSystemOnUpgrade`
2018-04-04 19:11:46 -04:00
Daniel Gasienica
4c4443390a Rename createImporter to createAttachmentDataWriter 2018-04-04 19:11:46 -04:00
Daniel Gasienica
5ea1a305a6 Clarify error origin 2018-04-04 18:41:50 -04:00
Daniel Gasienica
9dcb64b53c Initialize schema version upon import 2018-04-04 18:41:50 -04:00
Daniel Gasienica
94c75762b5 Add Message.createAttachmentLoader 2018-04-04 18:41:50 -04:00
Daniel Gasienica
bf67254cc5 Add Message.createImporter 2018-04-04 18:41:50 -04:00
Daniel Gasienica
efd673083d Rename writeAttachmentData to writeNewAttachmentData 2018-04-04 18:41:50 -04:00
Daniel Gasienica
d9be6a0f94 Destructure Lodash requires 2018-04-02 15:12:04 -04:00
Daniel Gasienica
5910f84af4 Remove outdated documentation 2018-04-02 12:17:36 -04:00
Daniel Gasienica
4651cbc2eb Improve `Message.upgradeSchema preconditions 2018-03-26 16:49:21 -04:00
Daniel Gasienica
8e762797c0 Remove attachment validation
Turns out `fileName` is optional and maybe even `contentType`. We should revisit
this at a later point in time.
2018-03-26 16:49:21 -04:00
Daniel Gasienica
867f73b80a Use double quotes for identifiers in error messages 2018-03-26 16:38:34 -04:00
Daniel Gasienica
2ad0fd7db6 Use Lodash omit instead of delete 2018-03-26 16:38:34 -04:00
Daniel Gasienica
279b3f81c7 Add Attachment.deleteData 2018-03-26 16:38:34 -04:00
Daniel Gasienica
9a540d6d18 Load attachment data before sending message 2018-03-26 16:38:34 -04:00
Daniel Gasienica
6355c54114 Return relative path from writeAttachmentData
This will make our app more robust to changes in location on the file system.
2018-03-26 16:38:34 -04:00
Daniel Gasienica
2cd3d5ac16 Add Message schema version 3 2018-03-26 16:38:34 -04:00
Daniel Gasienica
aa82a2a5fb Add context to Message.upgradeSchema
Allows us to pass in Electron/Node.js specific functions. This allows better
unit testing in Mocha where we don’t have access to Electron APIs.
2018-03-26 16:38:34 -04:00
Daniel Gasienica
ebe2a769c9 Add Attachment.migrateDataToFileSystem 2018-03-26 16:38:34 -04:00
Daniel Gasienica
a619d48fac Update schema version history 2018-03-19 19:27:59 -04:00
Daniel Gasienica
0e20e8e2ea Use Errors.toLogFormat 2018-03-19 19:27:59 -04:00
Daniel Gasienica
a5edbf8328 Initialize schema as first step in upgradeSchema 2018-03-19 19:27:59 -04:00
Daniel Gasienica
752c8f97e6 🎨 Format ternaries 2018-03-19 19:27:59 -04:00
Daniel Gasienica
8dfaa5619f Prefix private functions with underscore 2018-03-19 19:27:59 -04:00
Daniel Gasienica
ed336d31d7 Move schema versioning from Attachment to Message 2018-03-19 19:27:59 -04:00
Daniel Gasienica
b9e9f5e19a 🎨 withInheritedSchemaVersion to inheritSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica
df693ade7c Allow Attachment.fileName to be null 2018-03-19 19:27:59 -04:00
Daniel Gasienica
c27746b79e Add Message.withInheritedSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica
e9e46464c2 Add SchemaVersion type 2018-03-19 19:27:59 -04:00
Daniel Gasienica
add4b11df3 Skip invalid attachments and make function sync 2018-03-19 19:27:59 -04:00
Daniel Gasienica
3dfc823716 Add Attachment.removeSchemaVersion 2018-03-19 19:27:59 -04:00
Daniel Gasienica
43b47fd349 Avoid ensure-error for privacy reasons
Example:

```
> node
> const nonError = {foo: 'i-am-private'};
undefined

// before
> util.inspect(nonError);
'{ foo: \'i-am-private\' }'

// after
> nonError.toString()
'[object Object]'
>
```
2018-03-08 16:23:51 -05:00
Daniel Gasienica
44b81f68dd Remove privacy redaction from Errors.toLogFormat 2018-03-08 16:23:51 -05:00