Commit Graph

879 Commits

Author SHA1 Message Date
Kyle Zhao 0f1afad6da FIX: extracted theme JavaScripts for multisite (#6502)
* FIX: extracted theme javascripts for multisite

* onceoff to rebake all theme fields
2018-10-18 17:05:34 +11:00
Sam 8d06731484 FIX: reduce amount of work onceoff does
In the past onceoff was forcing inline download of gravatars,
this can be so expensive that it will never finish

This fix ensures it only marks avatars stale which will be picked
up by regular schedules
2018-10-16 10:29:16 +11:00
Guo Xiang Tan 84d4c81a26 FEATURE: Support backup uploads/downloads directly to/from S3.
This reverts commit 3c59106bac.
2018-10-15 09:43:31 +08:00
Régis Hanol 09961fb425 FIX: properly escape name of custom emoji 2018-10-11 09:35:23 +02:00
Guo Xiang Tan 3c59106bac Revert "FEATURE: Support backup uploads/downloads directly to/from S3."
This reverts commit c29a4dddc1.

We're doing a beta bump soon so un-revert this after that is done.
2018-10-11 11:08:23 +08:00
Gerhard Schlager c29a4dddc1 FEATURE: Support backup uploads/downloads directly to/from S3. 2018-10-11 10:38:43 +08:00
Bianca Nenciu e0e0a91e7d FIX: Retry sending email in case of temporary issue. (#6375) 2018-10-04 22:56:24 +08:00
Gerhard Schlager 3a6ab02563 FEATURE: Notify admins when scheduling of backup fails 2018-10-02 15:48:16 +02:00
Guo Xiang Tan d8fa7fbbab Fix onceoff job in cfa7173da3 not running. 2018-10-01 17:14:04 +08:00
Guo Xiang Tan cfa7173da3 FIX: Onceoff job to fix missing user profile backgrounds. 2018-10-01 16:26:40 +08:00
Robin Ward 02da022c70
PERF: Quit out of the email job quickly if disabled (#6423)
This prevents sidekiq from doing a bunch of queries when email is
disabled.

Critical emails are a special case and will be sent.
2018-10-01 01:15:45 +08:00
Guo Xiang Tan 767f27929d Rename `Jobs::RecoverPostUploads` to rerun the job take 2. 2018-09-19 22:40:32 +08:00
Guo Xiang Tan 6dbe1d832d Rename `Jobs::RecoverPostUploads` to rerun the job.
Running a migration to destroy the onceoff job doesn't work
because an instance running the old code may execute the job.
2018-09-19 21:59:44 +08:00
Guo Xiang Tan 667e68ec58 Prefer `create` over `create!` when not checking for response. 2018-09-19 18:22:54 +08:00
Guo Xiang Tan aa1af9fc22 FIX: Onceoff job to recover missing post uploads.
This fixes the regression due to 1f636c445b
2018-09-14 09:04:01 +08:00
Guo Xiang Tan e1b16e445e Rename `FileHelper.is_image?` -> `FileHelper.is_supported_image?`. 2018-09-12 09:22:28 +08:00
Gerhard Schlager 797cbf8653 FIX: Remove user fields when anonymizing user 2018-09-07 00:02:56 +02:00
Guo Xiang Tan b6a139b581 Fix broken spec. 2018-09-06 12:41:43 +08:00
Sam d9c0dc8687 correct prev commit
s3. did not exists it is s3-
2018-09-05 16:11:44 +10:00
Sam 83e1315e42 FIX: correct urls in uploads table to point at dualstack
Last week we added support for dual stack urls but did not remap the
the old records in the uploads and optimized images table

This caused a few minor edge cases worst was that if you rebaked old
images S3 CDN was not repopulated.
2018-09-05 15:58:04 +10:00
Sam e4498d2a8a FIX: keep db and job correctly in multisite logs
This ensures we report job and db correctly, previously we were
only reporting this on default
2018-09-04 16:05:44 +10:00
Sam e1975e293f FIX: when uploads are destroyed clear up avatar refs in user table
This also auto corrects twice daily when we ensure consistency
2018-08-31 14:46:42 +10:00
Guo Xiang Tan 4f81d4cc8e Minor fixes to `Jobs::BulkInvite`. 2018-08-30 15:35:16 +08:00
Gerhard Schlager b2cf725700 FIX: Don't try to send invite email when invite was deleted 2018-08-29 12:43:12 +02:00
Sam 44cf3cf975 FIX: queue heartbeats in readonly modes
If sidekiq is paused or Discourse is in readonly continue to queue
heartbeats

If we do not do that then a master process can end up reaping sidekiq
workers and causing various badness

This also impacts restore which can do weird stuff TM in cases like this
2018-08-29 12:36:59 +10:00
Sam 740308675b FEATURE: erode bounce score every time an email is sent
Introduces a hidden setting (default is 0.1) that erodes bounce score
every time we send an email. This means that erratic failures are less
painful cause system auto corrects
2018-08-28 17:02:12 +10:00
Sam 9ab1fb7dfc FEATURE: correctly store width and height on uploads
Previously we used width and height for thumbnails, new code ensures

1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
 by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions

Also

- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
2018-08-28 12:59:22 +10:00
Guo Xiang Tan 36a7028f19 FEATURE: Clean up `PostReplyKey` records.
* Default retention of 90 days.
2018-08-23 10:40:02 +08:00
Guo Xiang Tan 8bdf14834b PERF: Restrict number of skipped email log for `Jobs::UserEmail`. 2018-08-21 11:14:43 +08:00
Guo Xiang Tan 2c70d3f443 Take 2 on ba6f11c521. 2018-08-21 10:06:36 +08:00
Bianca Nenciu 860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Guo Xiang Tan 16c0ebe8a8 Fix the build. 2018-08-17 16:53:07 +08:00
Sam 796164b58c FIX: automatically correct bad avatars on access
Also start relying on upload extension for optimized images
2018-08-16 16:32:56 +10:00
Sam 1f17b84b63 FEATURE: more context for error reporting on jobs fails 2018-08-16 12:38:49 +10:00
Régis Hanol de92913bf4 FIX: store the topic links using the cooked upload url 2018-08-14 12:23:32 +02:00
Guo Xiang Tan ba022234c6 Add onceoff job to fix incorrect upload extensions. 2018-08-13 10:44:08 +08:00
Gerhard Schlager b73950692b FIX: Parsing non-existent feed should not fail 2018-08-10 18:37:14 +02:00
Simon Cossar 3cd4dc0f5f Allow users with group_locked_trust_level to be promoted to tl3 (#6249) 2018-08-10 10:42:23 +10:00
Guo Xiang Tan ba6f11c521 PERF: Only log the first skipped email when user exceeds daily limit.
https://meta.discourse.org/t/cleaning-up-e-mail-logs/39132
2018-08-08 16:25:00 +08:00
Guo Xiang Tan d5a7b818ba FIX: Move Jobs::CreateAvatarThumbnails to low priority queue. 2018-08-07 16:05:54 +08:00
Guo Xiang Tan 4e11811321 FIX: `UserAvatar#update_gravatar!` does not update `User#uploaded_avatar`.
https://meta.discourse.org/t/missing-user-profile-pictures/93844/4
2018-08-06 13:29:44 +08:00
Osama Sayegh 880462a41c FEATURE: display out of date themes on admin dashboard
* FEATURE: disaply out of date themes on admin dashboard

* Update copy
2018-08-03 09:53:48 +10:00
Neil Lalonde b829452c75
Merge pull request #6209 from discourse/mini_scheduler
REFACTOR: extract scheduler to the mini_scheduler gem
2018-08-01 10:28:24 -04:00
Joffrey JAFFEUX 0b9437cee7
FIX: more resilient/consistent dashboard caching (#6223) 2018-08-01 09:45:50 -04:00
Gerhard Schlager a115aae45f Use rchardet instead of charlock_holmes gem 2018-08-01 10:41:20 +02:00
Gerhard Schlager 5d421fb946 FIX: Try respecting charset in HTTP header of RSS feed 2018-08-01 10:41:20 +02:00
Gerhard Schlager ff942ed2f3 FIX: Try detecting encoding of RSS feed 2018-08-01 10:41:20 +02:00
Neil Lalonde 4ad7ce70ce REFACTOR: extract scheduler to the mini_scheduler gem 2018-07-31 17:12:55 -04:00
Guo Xiang Tan acde8d4323 Fix the build. 2018-07-30 15:07:03 +08:00
Guo Xiang Tan 96a0448c52 FIX: Add onceoff job to fix incorrect extension for gravatar uploads. 2018-07-30 14:44:02 +08:00
Guo Xiang Tan 87537b679c Drop `reply_key`, `skipped` and `skipped_reason` from `email_logs`. 2018-07-30 11:39:28 +08:00
Guo Xiang Tan ccf76d45f2 FIX: Missing variable outside of `begin` block. 2018-07-27 08:19:11 +08:00
Joffrey JAFFEUX 796639a797
FIX: makes disk_space computation more resilient (#6172) 2018-07-25 11:04:01 -04:00
Gerhard Schlager c3b6811651 PERF: Split loading of posts to speed up user renames 2018-07-24 11:57:04 +02:00
Guo Xiang Tan fad9c2b971 PERF: Move `EmailLog#reply_key` into new `post_reply_keys` table. 2018-07-24 13:51:53 +08:00
Guo Xiang Tan ae8b0a517f PERF: Split skipped email logs into a seperate table. 2018-07-24 13:14:37 +08:00
Vinoth Kannan f8e9190617 FEATURE: Retry web hook when it is failed 2018-07-23 10:12:04 +08:00
Joffrey JAFFEUX 06323f9c89
FIX: takes old dashboard out of caching job 2018-07-21 12:42:03 -04:00
Joffrey JAFFEUX 0972dd2552
FIX: takes dashboard-next out of caching job 2018-07-21 11:13:15 -04:00
Joffrey JAFFEUX 1d5096eb46 FIX: lazy load more reports in dashboard 2018-07-20 23:35:53 -04:00
Joffrey JAFFEUX 1a78e12f4e
FEATURE: part 2 of dashboard improvements
- moderation tab
- sorting/pagination
- improved third party reports support
- trending charts
- better perf
- many fixes
- refactoring
- new reports

Co-Authored-By: Simon Cossar <scossar@users.noreply.github.com>
2018-07-19 14:33:11 -04:00
Régis Hanol 6d6e026e3c FEATURE: selectable avatars 2018-07-18 12:57:43 +02:00
Sam 91266cdabb correct auto bump topic logic 2018-07-17 09:33:33 +10:00
Sam ac0053f491 FEATURE: navigate to first post and auto bump category settings
### navigate_to_first_post_after_read setting for categories

When enabled on categories logged on users will return to OP after
reading the entire category. (useful for documentation categories)

### num_auto_bump_daily

Set a number of topics that will automatically bump daily on a category.

- Every 15 minutes we will check if any category has this setting
- Categories with the setting are shuffled
- We exclude pinned, closed, category description and archived topics
- Maximum of 1 topic for the list of categories is bumped till limit reached per category
- We always try to bump oldest first
- Limit is elastic using a RateLimiter that ensures that we only bump N per day

Also some minor organisation on category settings

Froze strings on category.rb
2018-07-16 18:10:35 +10:00
Guo Xiang Tan 4172e1dd52
FIX: Rename `User#usernames` that clashes with `Group#name`. (#6069) 2018-07-09 16:54:57 +08:00
Guo Xiang Tan f7322c05ad FIX: Avoid `destroy_all` in `Jobs::CleanUpUploads`.
`destroy_all` loads all the relation into memory as once.

See https://github.com/rails/rails/issues/22510
2018-07-02 12:44:36 +08:00
Arpit Jalan 7550e9ff95 FIX: purge unactivated users with a message from non-human users 2018-06-29 13:03:04 +05:30
Ernesto Serrano c5e24da5dd Update poll_mailbox.rb 2018-06-25 16:18:07 +10:00
Sam cbaf521fc1 correct regression and add integrity spec for onceoffs 2018-06-20 09:09:31 +10:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam 12fd0b827e PERF: release results in a couple of cases 2018-06-13 15:11:43 +10:00
Gerhard Schlager 2564a8285d REFACTOR: Run some parts of user anonymizing in background job 2018-06-08 15:50:07 +02:00
Guo Xiang Tan ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Guo Xiang Tan d600e71b3d FIX: Clean up stale `UserExport` records daily.
* Add tests for `UserExport.remove_old_exports`
2018-06-05 09:43:40 +08:00
Régis Hanol 6024fb84cc make rubocop happy 2018-06-04 19:06:52 +02:00
Régis Hanol ab8cdd6b5b
Upload.url can't be NULL 2018-06-04 18:43:00 +02:00
Régis Hanol 085eaaf18d
FIX: always delete invalid upload records 2018-06-04 18:40:57 +02:00
Robin Ward 587a9c82f5 FIX: Use the username formatter when updating cooked usernames 2018-05-25 17:29:06 -04:00
Régis Hanol 71f66cd679 FIX: ensure PostAlerter is always run in sidekiq 2018-05-24 17:27:43 +02:00
Arpit Jalan cafd1241b9 FIX: make report data export more resilient to inconsistent date format
The DAU/MAU report export was failing because of date being a string
and not a valid Date value
2018-05-24 17:14:08 +05:30
Guo Xiang Tan 43f7cb05c9 FIX: Broken ping event for web hooks due to missing payload. 2018-05-24 15:16:52 +08:00
Gerhard Schlager 95db5f0c8c FIX: Do not replace wrong avatars when renaming user 2018-05-22 13:41:51 +02:00
Sam 24abf38d38 PERF: lower the priority on user emails
User emails can wait behind other regular jobs, they are usually somewhat
slower as they involve smtp transactions
2018-05-22 15:20:55 +10:00
Guo Xiang Tan bf84037f79 FIX: Payload for webhooks should be current as of the time the event was triggered.
https://meta.discourse.org/t/group-category-tag-user-deleted-webhooks-not-firing/87752
2018-05-21 17:29:58 +08:00
Arpit Jalan 1841dd48dc FIX: revert utf-8 encode changes 2018-05-20 17:50:36 +05:30
Arpit Jalan 9512796ef6 FIX: check for blank response when polling feed 2018-05-18 17:00:44 +05:30
Guo Xiang Tan 1ff767559d Fix incorrect param. 2018-05-18 14:05:45 +08:00
Guo Xiang Tan c42b65df5f `find` raises an error if the record is missing. 2018-05-18 13:37:07 +08:00
Arpit Jalan 238a13643d FIX: handle missing users when sending push notifications 2018-05-17 12:53:19 +05:30
Régis Hanol 39aceed63d FIX: don't purge unactivated users with a message 2018-05-16 18:24:11 +02:00
Arpit Jalan 146f634c8f FIX: UTF-8 encode feed response body 2018-05-16 12:59:23 +05:30
Sam 21e0b7c818 avoid async report pattern and replace with simpler hijack 2018-05-16 16:05:03 +10:00
Gerhard Schlager 74c4af279a Improvements to user renaming
* don't update search index if post belongs to deleted topic
* log errors instead of crashing when updating post or revision fails
* update mentions even when the href attribute is missing
* run the background job with low priority
* replace username in all notifications
* update `action_code_who` used by small action posts
2018-05-15 21:05:51 +02:00
Gerhard Schlager 2e1b5bc8d3 FIX: Transaction in UserAnonymizer prevented avatar from updating 2018-05-15 20:47:58 +02:00
Joffrey JAFFEUX 9947c38e1c
UX: support for multiple datasets in one chart 2018-05-15 20:12:03 +02:00
Sam 193b6d5651 UX: improve new dashboard
- top referred topics
- limit search logs to 8 results
2018-05-15 15:08:36 +10:00
Régis Hanol a28c58feb1 FIX: automatic group membership when using SSO 2018-05-15 01:48:30 +02:00
Gerhard Schlager c67c2dc638 FIX: Username update should ignore revisions without raw 2018-05-14 15:22:42 +02:00
Gerhard Schlager 8232aba743 FIX: Update search index after renaming user within posts 2018-05-14 13:20:30 +02:00
Gerhard Schlager 02cb84847c FIX: Update avatar in oneboxed posts when renaming user 2018-05-14 13:20:29 +02:00