Commit Graph

1061 Commits

Author SHA1 Message Date
Sam d9c0dc8687 correct prev commit
s3. did not exists it is s3-
2018-09-05 16:11:44 +10:00
Sam 83e1315e42 FIX: correct urls in uploads table to point at dualstack
Last week we added support for dual stack urls but did not remap the
the old records in the uploads and optimized images table

This caused a few minor edge cases worst was that if you rebaked old
images S3 CDN was not repopulated.
2018-09-05 15:58:04 +10:00
Sam e4498d2a8a FIX: keep db and job correctly in multisite logs
This ensures we report job and db correctly, previously we were
only reporting this on default
2018-09-04 16:05:44 +10:00
Sam e1975e293f FIX: when uploads are destroyed clear up avatar refs in user table
This also auto corrects twice daily when we ensure consistency
2018-08-31 14:46:42 +10:00
Guo Xiang Tan 4f81d4cc8e Minor fixes to `Jobs::BulkInvite`. 2018-08-30 15:35:16 +08:00
Gerhard Schlager b2cf725700 FIX: Don't try to send invite email when invite was deleted 2018-08-29 12:43:12 +02:00
Sam 44cf3cf975 FIX: queue heartbeats in readonly modes
If sidekiq is paused or Discourse is in readonly continue to queue
heartbeats

If we do not do that then a master process can end up reaping sidekiq
workers and causing various badness

This also impacts restore which can do weird stuff TM in cases like this
2018-08-29 12:36:59 +10:00
Sam 740308675b FEATURE: erode bounce score every time an email is sent
Introduces a hidden setting (default is 0.1) that erodes bounce score
every time we send an email. This means that erratic failures are less
painful cause system auto corrects
2018-08-28 17:02:12 +10:00
Sam 9ab1fb7dfc FEATURE: correctly store width and height on uploads
Previously we used width and height for thumbnails, new code ensures

1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
 by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions

Also

- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
2018-08-28 12:59:22 +10:00
Guo Xiang Tan 36a7028f19 FEATURE: Clean up `PostReplyKey` records.
* Default retention of 90 days.
2018-08-23 10:40:02 +08:00
Guo Xiang Tan 8bdf14834b PERF: Restrict number of skipped email log for `Jobs::UserEmail`. 2018-08-21 11:14:43 +08:00
Guo Xiang Tan 2c70d3f443 Take 2 on ba6f11c521. 2018-08-21 10:06:36 +08:00
Bianca Nenciu 860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Guo Xiang Tan 16c0ebe8a8 Fix the build. 2018-08-17 16:53:07 +08:00
Sam 796164b58c FIX: automatically correct bad avatars on access
Also start relying on upload extension for optimized images
2018-08-16 16:32:56 +10:00
Sam 1f17b84b63 FEATURE: more context for error reporting on jobs fails 2018-08-16 12:38:49 +10:00
Régis Hanol de92913bf4 FIX: store the topic links using the cooked upload url 2018-08-14 12:23:32 +02:00
Guo Xiang Tan ba022234c6 Add onceoff job to fix incorrect upload extensions. 2018-08-13 10:44:08 +08:00
Gerhard Schlager b73950692b FIX: Parsing non-existent feed should not fail 2018-08-10 18:37:14 +02:00
Simon Cossar 3cd4dc0f5f Allow users with group_locked_trust_level to be promoted to tl3 (#6249) 2018-08-10 10:42:23 +10:00
Guo Xiang Tan ba6f11c521 PERF: Only log the first skipped email when user exceeds daily limit.
https://meta.discourse.org/t/cleaning-up-e-mail-logs/39132
2018-08-08 16:25:00 +08:00
Guo Xiang Tan d5a7b818ba FIX: Move Jobs::CreateAvatarThumbnails to low priority queue. 2018-08-07 16:05:54 +08:00
Guo Xiang Tan 4e11811321 FIX: `UserAvatar#update_gravatar!` does not update `User#uploaded_avatar`.
https://meta.discourse.org/t/missing-user-profile-pictures/93844/4
2018-08-06 13:29:44 +08:00
Osama Sayegh 880462a41c FEATURE: display out of date themes on admin dashboard
* FEATURE: disaply out of date themes on admin dashboard

* Update copy
2018-08-03 09:53:48 +10:00
Neil Lalonde b829452c75
Merge pull request #6209 from discourse/mini_scheduler
REFACTOR: extract scheduler to the mini_scheduler gem
2018-08-01 10:28:24 -04:00
Joffrey JAFFEUX 0b9437cee7
FIX: more resilient/consistent dashboard caching (#6223) 2018-08-01 09:45:50 -04:00
Gerhard Schlager a115aae45f Use rchardet instead of charlock_holmes gem 2018-08-01 10:41:20 +02:00
Gerhard Schlager 5d421fb946 FIX: Try respecting charset in HTTP header of RSS feed 2018-08-01 10:41:20 +02:00
Gerhard Schlager ff942ed2f3 FIX: Try detecting encoding of RSS feed 2018-08-01 10:41:20 +02:00
Neil Lalonde 4ad7ce70ce REFACTOR: extract scheduler to the mini_scheduler gem 2018-07-31 17:12:55 -04:00
Guo Xiang Tan acde8d4323 Fix the build. 2018-07-30 15:07:03 +08:00
Guo Xiang Tan 96a0448c52 FIX: Add onceoff job to fix incorrect extension for gravatar uploads. 2018-07-30 14:44:02 +08:00
Guo Xiang Tan 87537b679c Drop `reply_key`, `skipped` and `skipped_reason` from `email_logs`. 2018-07-30 11:39:28 +08:00
Guo Xiang Tan ccf76d45f2 FIX: Missing variable outside of `begin` block. 2018-07-27 08:19:11 +08:00
Joffrey JAFFEUX 796639a797
FIX: makes disk_space computation more resilient (#6172) 2018-07-25 11:04:01 -04:00
Gerhard Schlager c3b6811651 PERF: Split loading of posts to speed up user renames 2018-07-24 11:57:04 +02:00
Guo Xiang Tan fad9c2b971 PERF: Move `EmailLog#reply_key` into new `post_reply_keys` table. 2018-07-24 13:51:53 +08:00
Guo Xiang Tan ae8b0a517f PERF: Split skipped email logs into a seperate table. 2018-07-24 13:14:37 +08:00
Vinoth Kannan f8e9190617 FEATURE: Retry web hook when it is failed 2018-07-23 10:12:04 +08:00
Joffrey JAFFEUX 06323f9c89
FIX: takes old dashboard out of caching job 2018-07-21 12:42:03 -04:00
Joffrey JAFFEUX 0972dd2552
FIX: takes dashboard-next out of caching job 2018-07-21 11:13:15 -04:00
Joffrey JAFFEUX 1d5096eb46 FIX: lazy load more reports in dashboard 2018-07-20 23:35:53 -04:00
Joffrey JAFFEUX 1a78e12f4e
FEATURE: part 2 of dashboard improvements
- moderation tab
- sorting/pagination
- improved third party reports support
- trending charts
- better perf
- many fixes
- refactoring
- new reports

Co-Authored-By: Simon Cossar <scossar@users.noreply.github.com>
2018-07-19 14:33:11 -04:00
Régis Hanol 6d6e026e3c FEATURE: selectable avatars 2018-07-18 12:57:43 +02:00
Sam 91266cdabb correct auto bump topic logic 2018-07-17 09:33:33 +10:00
Sam ac0053f491 FEATURE: navigate to first post and auto bump category settings
### navigate_to_first_post_after_read setting for categories

When enabled on categories logged on users will return to OP after
reading the entire category. (useful for documentation categories)

### num_auto_bump_daily

Set a number of topics that will automatically bump daily on a category.

- Every 15 minutes we will check if any category has this setting
- Categories with the setting are shuffled
- We exclude pinned, closed, category description and archived topics
- Maximum of 1 topic for the list of categories is bumped till limit reached per category
- We always try to bump oldest first
- Limit is elastic using a RateLimiter that ensures that we only bump N per day

Also some minor organisation on category settings

Froze strings on category.rb
2018-07-16 18:10:35 +10:00
Guo Xiang Tan 4172e1dd52
FIX: Rename `User#usernames` that clashes with `Group#name`. (#6069) 2018-07-09 16:54:57 +08:00
Guo Xiang Tan f7322c05ad FIX: Avoid `destroy_all` in `Jobs::CleanUpUploads`.
`destroy_all` loads all the relation into memory as once.

See https://github.com/rails/rails/issues/22510
2018-07-02 12:44:36 +08:00
Arpit Jalan 7550e9ff95 FIX: purge unactivated users with a message from non-human users 2018-06-29 13:03:04 +05:30
Ernesto Serrano c5e24da5dd Update poll_mailbox.rb 2018-06-25 16:18:07 +10:00
Sam cbaf521fc1 correct regression and add integrity spec for onceoffs 2018-06-20 09:09:31 +10:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Sam 12fd0b827e PERF: release results in a couple of cases 2018-06-13 15:11:43 +10:00
Gerhard Schlager 2564a8285d REFACTOR: Run some parts of user anonymizing in background job 2018-06-08 15:50:07 +02:00
Guo Xiang Tan ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Guo Xiang Tan d600e71b3d FIX: Clean up stale `UserExport` records daily.
* Add tests for `UserExport.remove_old_exports`
2018-06-05 09:43:40 +08:00
Régis Hanol 6024fb84cc make rubocop happy 2018-06-04 19:06:52 +02:00
Régis Hanol ab8cdd6b5b
Upload.url can't be NULL 2018-06-04 18:43:00 +02:00
Régis Hanol 085eaaf18d
FIX: always delete invalid upload records 2018-06-04 18:40:57 +02:00
Robin Ward 587a9c82f5 FIX: Use the username formatter when updating cooked usernames 2018-05-25 17:29:06 -04:00
Régis Hanol 71f66cd679 FIX: ensure PostAlerter is always run in sidekiq 2018-05-24 17:27:43 +02:00
Arpit Jalan cafd1241b9 FIX: make report data export more resilient to inconsistent date format
The DAU/MAU report export was failing because of date being a string
and not a valid Date value
2018-05-24 17:14:08 +05:30
Guo Xiang Tan 43f7cb05c9 FIX: Broken ping event for web hooks due to missing payload. 2018-05-24 15:16:52 +08:00
Gerhard Schlager 95db5f0c8c FIX: Do not replace wrong avatars when renaming user 2018-05-22 13:41:51 +02:00
Sam 24abf38d38 PERF: lower the priority on user emails
User emails can wait behind other regular jobs, they are usually somewhat
slower as they involve smtp transactions
2018-05-22 15:20:55 +10:00
Guo Xiang Tan bf84037f79 FIX: Payload for webhooks should be current as of the time the event was triggered.
https://meta.discourse.org/t/group-category-tag-user-deleted-webhooks-not-firing/87752
2018-05-21 17:29:58 +08:00
Arpit Jalan 1841dd48dc FIX: revert utf-8 encode changes 2018-05-20 17:50:36 +05:30
Arpit Jalan 9512796ef6 FIX: check for blank response when polling feed 2018-05-18 17:00:44 +05:30
Guo Xiang Tan 1ff767559d Fix incorrect param. 2018-05-18 14:05:45 +08:00
Guo Xiang Tan c42b65df5f `find` raises an error if the record is missing. 2018-05-18 13:37:07 +08:00
Arpit Jalan 238a13643d FIX: handle missing users when sending push notifications 2018-05-17 12:53:19 +05:30
Régis Hanol 39aceed63d FIX: don't purge unactivated users with a message 2018-05-16 18:24:11 +02:00
Arpit Jalan 146f634c8f FIX: UTF-8 encode feed response body 2018-05-16 12:59:23 +05:30
Sam 21e0b7c818 avoid async report pattern and replace with simpler hijack 2018-05-16 16:05:03 +10:00
Gerhard Schlager 74c4af279a Improvements to user renaming
* don't update search index if post belongs to deleted topic
* log errors instead of crashing when updating post or revision fails
* update mentions even when the href attribute is missing
* run the background job with low priority
* replace username in all notifications
* update `action_code_who` used by small action posts
2018-05-15 21:05:51 +02:00
Gerhard Schlager 2e1b5bc8d3 FIX: Transaction in UserAnonymizer prevented avatar from updating 2018-05-15 20:47:58 +02:00
Joffrey JAFFEUX 9947c38e1c
UX: support for multiple datasets in one chart 2018-05-15 20:12:03 +02:00
Sam 193b6d5651 UX: improve new dashboard
- top referred topics
- limit search logs to 8 results
2018-05-15 15:08:36 +10:00
Régis Hanol a28c58feb1 FIX: automatic group membership when using SSO 2018-05-15 01:48:30 +02:00
Gerhard Schlager c67c2dc638 FIX: Username update should ignore revisions without raw 2018-05-14 15:22:42 +02:00
Gerhard Schlager 8232aba743 FIX: Update search index after renaming user within posts 2018-05-14 13:20:30 +02:00
Gerhard Schlager 02cb84847c FIX: Update avatar in oneboxed posts when renaming user 2018-05-14 13:20:29 +02:00
Sam 6332d5040d UX: switch dashboard to be the new dashboard
Also:
- add pageviews
- add problems and version sections
2018-05-14 13:07:59 +10:00
Sam bc9e0d46af PERF: use cached reports for dashboard if available 2018-05-14 12:01:44 +10:00
Sam 9d97e1244e correct it so when we have no dates we still return graphs 2018-05-14 11:12:52 +10:00
Sam 67054d524d correct missing day from async reports 2018-05-11 15:06:23 +10:00
Sam 8a783412b7 UX: improvements to new dashboard
- remove inactive user report and replace with posts
- clean up internals so grouping by week happens on client
- when switching periods old report was not destroyed leading to bugs
- calculate trend based on previous interval ... not previous 30 days
- show percentages for mau/dau
- be more careful about utc date usage
- show uniqu and click through rate on search panel
- publish key of report with report so we only load the correct one
- subscribe earlier in channel in case of concurrency issues
2018-05-11 13:30:32 +10:00
Gerhard Schlager 2e67998319 Improvements for user renaming (#5810)
* FEATURE: Update avatars in posts and revisions when user gets renamed

* FIX: Replace username in deleted posts when user gets renamed

* FEATURE: Replace username in notifications when user gets renamed

FEATURE: Update mentions and quotes when user gets merged
2018-05-08 10:02:43 -04:00
Misaka 0x4e21 ff6be3c2e3 FEATURE: add profile_background fields into SSO (#5701)
Add profile_background and card_background fields into Discourse SSO.
2018-05-07 10:03:26 +02:00
Jeff Wong 91b31860a1
Feature: Push notifications for Android (#5792)
* Feature: Push notifications for Android

Notification config for desktop and mobile are merged.

Desktop notifications stay as they are for desktop views.

If mobile mode, push notifications are enabled.

Added push notification subscriptions in their own table, rather than through
custom fields.

Notification banner prompts appear for both mobile and desktop when enabled.
2018-05-04 15:31:48 -07:00
Joffrey JAFFEUX 980972182f
dashboard next: caching, mobile support and new charts 2018-05-03 15:41:41 +02:00
Gerhard Schlager 3be3c50c7e FEATURE: Rename user in mentions and quotes
Co-authored-by: Robin Ward <robin.ward@gmail.com>
2018-05-01 13:49:14 -04:00
Arpit Jalan 4f55fbfefa FEATURE: include report title in PM subject and filename 2018-04-24 22:25:54 +05:30
Arpit Jalan a1f0c58f23 FIX: export download link was broken 2018-04-20 01:12:41 +05:30
Arpit Jalan 91bf10bd12 FIX: create upload record for exported csv files 2018-04-20 00:27:49 +05:30
Joffrey JAFFEUX 0e414d0890
dashboard next: trending search report
This commit also improves how data is loaded sync and async
2018-04-19 18:19:21 +02:00
Joffrey JAFFEUX 01c061d20d
dashboard next: perf and UI tweaks
* cache CORE reports
* adds backups/uploads section
* few css tweaks
2018-04-18 21:30:41 +02:00
Guo Xiang Tan 7bf9650e96 Remove comment that is no longer accurate. 2018-04-18 14:22:18 +08:00
Arpit Jalan a16b616861 FEATURE: webhook for flag events 2018-04-13 07:47:58 +05:30
Arpit Jalan f5febe5843 FIX: date range was inconsistent for admin dashboard reports 2018-04-08 16:49:41 +05:30
Vinoth Kannan 434cbc649f FEATURE: Webhook for tag events 2018-04-04 17:49:20 +05:30
Guo Xiang Tan 142571bba0 Remove use of `rescue nil`.
* `rescue nil` is a really bad pattern to use in our code base.
  We should rescue errors that we expect the code to throw and
  not rescue everything because we're unsure of what errors the
  code would throw. This would reduce the amount of pain we face
  when debugging why something isn't working as expexted. I've
  been bitten countless of times by errors being swallowed as a
  result during debugging sessions.
2018-04-02 13:52:51 +08:00
Vinoth Kannan efb19dbdaf
Merge pull request #5705 from discourse/new_webhooks
FEATURE: Webhook for group and category events
2018-04-02 10:53:21 +05:30
Guo Xiang Tan 90f91bf017 Fix regression due to ee69d58a59. 2018-03-29 10:01:29 +08:00
Vinoth Kannan dc33f2d071 Add new web hook serializers 2018-03-28 17:40:29 +05:30
Guo Xiang Tan ee69d58a59 FIX: Tests could get stucked in infinite loop if it fails to resolve IP of a hostname. 2018-03-28 14:49:05 +08:00
Guo Xiang Tan 347e4eadbc Don't retry trying to download a file in test. 2018-03-28 12:54:11 +08:00
Neil Lalonde 7311023a52
Merge pull request #5700 from discourse/crawl-block
FEATURE: control web crawlers access with white/blacklist
2018-03-27 15:06:03 -04:00
Guo Xiang Tan 2f61780a49 Remove file that is no longer used. 2018-03-26 10:27:28 +08:00
Neil Lalonde ced7e9a691 FEATURE: control which web crawlers can access using a whitelist or blacklist 2018-03-22 15:41:02 -04:00
Robin Ward b9abd7dc9e FEATURE: Shared Drafts
This feature can be enabled by choosing a destination for the
`shared drafts category` site setting.

* Staff members can create shared drafts, choosing a destination
category for the topic when it is published.

* Shared Drafts can be viewed in their category, or above the
topic list for the destination category where it will end up.

* When the shared draft is ready, it can be published to the
appropriate category by clicking a button on the topic view.

* When published, Drafts change their timestamps to the current
time, and any edits to the original post are removed.
2018-03-20 17:15:26 -04:00
Guo Xiang Tan 486bbe9cc2 FIX: Auto re-opened topics should restore category auto close settings.
https://meta.discourse.org/t/topic-closed-temporarily-due-to-community-flags-overwrites-category-auto-close-timer/77421
2018-02-27 15:33:31 +08:00
Sam 86d12bd44b FEATURE: search within title using in:title
Also

- Significantly improved search ranking, title is treated most strongly
- Adds tag names to the index
- Run search re-indexer more aggressively
- Re-index topic and all posts on category change
2018-02-20 14:41:21 +11:00
Arpit Jalan 33df2d6a02 FIX: data export should fill missing dates with zero value 2018-02-18 23:52:09 +05:30
Neil Lalonde 991dfadad7 FIX: CategoryTagStat queries need to exclude PMs with tags 2018-02-15 12:13:42 -05:00
Neil Lalonde fdd8ce9722 empty category_tag_stats table before running init job 2018-02-15 11:28:25 -05:00
Sam d8b4627fc8 we have to define this for tests to pass 2018-02-15 13:30:34 +11:00
Sam b5b866aab3 oops 2018-02-15 13:13:31 +11:00
Sam c89b42c488 PERF: only require the rss library if used
Before:

Total allocated: 257909321 bytes (2514134 objects)
Total retained:  39681579 bytes (343387 objects)

allocated memory by gem
-----------------------------------
  42875979  rss

retained memory by gem
-----------------------------------
   2080188  rss

retained objects by gem
-----------------------------------
     13052  rss

After:

Total allocated: 210562047 bytes (2252030 objects)
Total retained:  37433816 bytes (328635 objects)

----

So, 2 less megabytes on boot and 13000 objects stuck in ruby heaps forever.
2018-02-15 13:11:33 +11:00
Régis Hanol 7d501058d7 FIX: don't grant new user of the month badge to suspended users or based on deleted topics/posts 2018-02-13 17:32:38 +01:00
Neil Lalonde 76c309fe6b PERF: a faster way to count tags used per category 2018-02-12 15:16:47 -05:00
Vinoth Kannan e700e3e882 FIX: post_destroyed webhook event not fired 2018-02-13 01:36:56 +05:30
Neil Lalonde d7e90edeb5 PERF: Tl3Promotions job can limit the number of TL3 candidates by using some simple requirements in the query 2018-02-05 17:54:36 -05:00
Neil Lalonde 8fe61c68a1 FIX: process_post job should update baked_at and baked_version if it rebakes the post 2018-02-05 11:47:04 -05:00
Régis Hanol 130974d58d FIX: automatically unsilence users 2018-02-05 17:45:47 +01:00
Régis Hanol d233ecbe34 push updates to backups list to client 2018-01-31 12:05:06 +01:00
Régis Hanol 77ac14d475 prevent users with a tiny number of posts to dominate the 'new user of the month' leaderboard 2018-01-29 12:09:17 +01:00
Sam 47058b29d3 FEATURE: emit external_id field with user webhook 2018-01-22 10:09:26 +11:00
Neil Lalonde 2493648f9c PERF: calculate topic_counts for tags in an async job so tag queries that include counts are much faster 2018-01-12 11:03:03 -05:00
Jan Suchal bc56d86a63 Support ruby 2.5.0 2018-01-09 16:03:17 +01:00
Robin Ward 69a90f31fb FEATURE: Allow Forums to disable the Backups feature 2017-12-21 15:22:04 -05:00
Gerhard Schlager 7b58afe677 FIX: ProcessPost job failed for posts that have no user 2017-12-21 14:45:59 +01:00
Sam 88a4ec5f1b FIX: stop forking regular backup jobs 2017-12-21 09:00:48 +11:00
Neil Lalonde f5a3be750c PERF: add option to limit how many categories are processed in one call to CategoryFeaturedTopic.feature_topics (#5446) 2017-12-20 13:42:29 +11:00
Guo Xiang Tan 97ceebb570 SECURITY: Don't pass email backup token to sidekiq as a parameter.
* This exposes the token in the Sidekiq dashboard which can be
  viewed by an admin and defeats the purpose of using a token
  in the download backup email ink.
2017-12-18 11:25:22 +08:00
Guo Xiang Tan 5489cd14c7 Skip validations when fixing `Topic#featured_link` in onceoff job. 2017-12-15 23:45:07 +08:00
scossar 11050e5d10 Don't override count value that has been set by query.count 2017-12-13 18:48:46 -08:00
Sam b998efdc94 FIX: do not send mailing list emails to unapproved users 2017-12-13 15:13:17 +11:00
Guo Xiang Tan 4bd5acec47 FIX: `Topic#featured_link` may contain more than a URL. 2017-12-11 16:36:19 +08:00
Arpit Jalan 3c56c9b637 FIX: strip webhook payload_url 2017-12-11 13:48:11 +05:30
Kyle Zhao 5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Neil Lalonde b35960fc35 FIX: export admin reports to csv would have 0 rows in the csv file 2017-12-01 15:50:36 -05:00
Guo Xiang Tan c128e421c4 FIX: Don't run job if topic timer has already been deleted. 2017-11-30 15:26:26 +08:00
Robin Ward 77f90876d3 REFACTOR: Track manual locked user levels separately from groups 2017-11-27 11:23:44 -05:00
Régis Hanol 4addc5e329 Add missing contexts when destroying users 2017-11-22 15:43:54 +01:00
Régis Hanol c4d0eb802e FIX: delete upload records when sha is null 2017-11-21 10:20:42 +01:00
Régis Hanol 2d48caffdf FIX: be more lenient when deleting a custom emoji 2017-11-20 23:50:23 +01:00
Neil Lalonde dcd60dcc8f FIX: cap posts_read_count in user_stats to 50 posts per topic_entered 2017-11-20 17:15:05 -05:00
Neil Lalonde b37e40eea9 FEATURE: show read time in last 60 days 2017-11-16 15:46:51 -05:00
Régis Hanol 678e28794a FIX: properly handle too large & broken images in posts 2017-11-16 15:45:07 +01:00
Sam 9c22c68d39 FIX: only save custom fields if they actually change 2017-11-16 15:14:10 +11:00
Vinoth Kannan 7b494a65c9 NEW: large image placeholder added in cooked html (#5291) 2017-11-15 11:30:47 +01:00
Robin Ward 971e302ff2 FEATURE: Support an end date for user silencing 2017-11-14 13:20:19 -05:00
Régis Hanol ba2209f7d7 FIX: always clean up uploads with no sha1 2017-11-14 10:56:10 +01:00
Régis Hanol 7370adeae3 FIX: don't delete uploads referenced in drafts or queued posts when using the short_url 2017-11-13 15:01:31 +01:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Guo Xiang Tan 6def5a344a Let's be more aggressive with purging old WebHookEvent records. 2017-11-08 14:15:34 +08:00
Guo Xiang Tan 6090994cdf FEATURE: Retain the latest 30 days of WebHookEvent records by default. 2017-11-08 14:11:01 +08:00
Sam 7c5a71e929 DEV: allow queue_jobs = false in dev
your mileage may vary
2017-10-31 13:50:58 +11:00
Guo Xiang Tan 0abc5f90cd FIX: Broken link in new user of the month post. 2017-10-30 13:11:29 +08:00
Neil Lalonde 2c15e9c6fe FIX: search couldn't find tags from before 2017-08-25 2017-10-24 17:55:05 -04:00
Guo Xiang Tan d5e369d6cf Fix weird Rubocop error. 2017-10-23 12:30:28 +08:00
Régis Hanol c838f43a75 let's not generate an error when logging errors... 2017-10-18 23:14:13 +02:00
Neil Lalonde c53f41f4f2 FIX: don't flag watched words when rebaking posts 2017-10-12 15:34:22 -04:00
Arpit Jalan 97395ebb66 FIX: SSO avatar downloads were broken
cc @tgxworld
2017-10-12 12:12:04 +05:30
Régis Hanol f7282e4ecd use force_https site setting when adding scheme for downloading schemaless images locally 2017-10-12 00:06:24 +02:00
Régis Hanol 4e78abb537 let's try 3 times to download images locally 2017-10-11 23:11:44 +02:00
Guo Xiang Tan 9dcb11f553 Fix the build. 2017-10-11 17:45:19 +08:00
Guo Xiang Tan 09721090a3 FIX: Ensure that we revert back to default connection after running jobs. 2017-10-11 17:17:03 +08:00
Guo Xiang Tan 59aeb0bc56 FIX: Sidekiq hot reloading wasn't working in dev.
https://meta.discourse.org/t/webhooks-sidekiq-issue-on-dev-instance/71129

* Remove code that is no longer required as well.
2017-10-09 18:23:25 +08:00
Sam 4ea87b5ab8 Merge branch 's3_refactor' 2017-10-09 10:27:52 +11:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan 4ba5e678d8 Require dependencies to enable live reload in dev for Sidekiq. 2017-10-06 11:39:00 +08:00
Régis Hanol e212435545 FIX: redirect to top wasn't working 2017-10-04 22:08:41 +02:00
Kyle Zhao ac666ddf17 PollFeed: check 'content:encoded' for content first 2017-10-02 01:16:11 -04:00
Guo Xiang Tan 0f2c5f5fc9 FIX: Don't raise error when trying to download avatar from URL. 2017-10-02 12:59:41 +08:00
Guo Xiang Tan 974836962d Fix invalid method call. 2017-10-02 12:50:22 +08:00
Guo Xiang Tan 77ea063751 FIX: Missing attribute. 2017-10-02 10:24:37 +08:00
Guo Xiang Tan 4eeb6014f4 Don't raise an error if user has been destroyed. 2017-09-30 09:09:40 +08:00
Guo Xiang Tan d79fee9ff3 Fix undefined method for Nil class error. 2017-09-28 17:38:53 +08:00
Sam 8ecf313a81 FIX: correctly raise errors when downloads fail
This corrects an issue where we are hitting Gravatar for 404 over and over

Also ensures file download properly reports errors
2017-09-28 16:35:43 +10:00
Régis Hanol 3a75242c38 add more logs to pull hotlinked images 2017-09-28 01:00:13 +02:00
Régis Hanol dd07094bd7 bump log level to debug download issues 2017-09-27 23:26:07 +02:00
Robin Ward fa41913ba5 FIX: Don't count private content in New User of the Month 2017-09-27 15:03:24 -04:00
Robin Ward d1ebc62065 The ability to display errors on flagging actions. 2017-09-25 12:28:01 -04:00
Robin Ward 677b016387 Send a suspension message via email to a user 2017-09-25 12:26:41 -04:00
Guo Xiang Tan 23b787e0a6 Require dependency otherwise it causes Sidekiq to lock up in development. 2017-09-25 13:48:59 +08:00
Neil Lalonde 16fe7aa307 FEATURE: automatically handle flags and posts that have been waiting in a queue for a long time. Flags will be deferred. Posts waiting for approval will be rejected. Control how old the records need to be with the auto_handle_queued_age site setting. 2017-09-14 12:01:06 -04:00
Régis Hanol d9465bac13 add 'staged' column to user export 2017-09-13 18:09:11 +02:00
Vinoth Kannan 6e9671c2c3 UX: Placeholder images color changed & tootip added 2017-09-13 15:16:38 +05:30
Neil Lalonde beea5cac48 FIX: send the queued posts reminder as a message to moderators instead of an email to the contact_email 2017-09-12 18:00:51 -04:00
Neil Lalonde 6831efe2e9 FIX: no notification was being sent when a post is hidden by community flags 2017-09-12 15:43:44 -04:00
Vinoth Kannan 0fb7831749 FEATURE: Add placeholders to broken and large image files (#5113) 2017-09-01 10:26:13 -04:00
Neil Lalonde 2c56f8df7c FEATURE: show tags in search results 2017-08-25 11:52:59 -04:00
Leo McArdle 3986367f3f update pr based on review 2017-08-23 14:55:34 +01:00
Leo McArdle be1df3ba75 FIX: transfer posts by duplicated staged users to original 2017-08-22 09:58:51 +01:00
Sam Saffron 56f7b4e01e PERF: reindex search data without loading large post counts 2017-08-16 08:18:59 -04:00
Erick Guan 6e59149a77 FIX: rebuild index when engine replaced (#5021) 2017-08-16 07:38:34 -04:00
Guo Xiang Tan b77aa29e71 Merge pull request #5013 from LeoMcA/alternate-emails-phase-1.5
FIX: add additional email to tests and clean up resulting mess
2017-08-16 16:19:28 +09:00
Guo Xiang Tan 793eccf6c5 FIX: Staged users were incorrectly created multiple times with same email. 2017-08-10 18:56:15 +09:00
Guo Xiang Tan a849959915 Fix missing parameter. 2017-08-10 11:15:51 +09:00
Guo Xiang Tan b404a4b97c Merge pull request #5034 from tgxworld/fix_staged_primary_email
FIX: Staged users are still missing primary email.
2017-08-10 10:30:51 +09:00
Arpit Jalan e993d53260 FIX: handle missing parent category when exporting user archive
https://meta.discourse.org/t/download-my-posts-failed/67613?u=techapj
2017-08-09 14:37:21 +05:30
Guo Xiang Tan 0bc690ed11 FIX: Staged users are still missing primary email. 2017-08-09 12:03:49 +09:00
Leo McArdle 836dee1120 FIX: add additional email to tests and clean up resulting mess 2017-07-31 22:27:29 +00:00
Neil Lalonde 5d528f0d15 Merge pull request #4958 from dmacjam/search_posts_by_filetype
FEATURE: Search posts by filetype
2017-07-31 11:55:34 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Neil Lalonde 24cb950432 FEATURE: Watched Words: when posts contain words, do one of flag, require approval, censor, or block 2017-07-26 11:01:09 -04:00
Neil Lalonde d8c27e3871 Merge branch 'master' into search_posts_by_filetype 2017-07-25 14:41:20 -04:00
Robin Ward 2f8f2aa1dd FEATURE: Whitelists for inline oneboxing 2017-07-21 15:41:47 -04:00
Leo McArdle ddc0134b48 add DiscourseEvent triggers necessary to update a user's permissions before they're notified 2017-07-21 11:03:54 -04:00
Leo McArdle d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Robin Ward 6b6ad9391b Clean up job for search logs 2017-07-14 14:30:58 -04:00
Jakub Macina 677267ae78 Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests. 2017-07-12 17:19:27 +02:00
Régis Hanol a6dff79c2c change log level to info when failing to download a hotlinked image 2017-07-12 11:06:28 +02:00
Guo Xiang Tan e92acb4c40 FIX: `Jobs::PullHotlinkedImages#is_valid_image_src` returns true for a generic string. 2017-07-06 18:31:15 +09:00
Guo Xiang Tan 061aa261da Log site name when logging to Logster in `Jobs::PullHotlinkedImages`. 2017-07-05 10:34:24 +09:00
Guo Xiang Tan 4e8b80c157 Remove unnecessary `ensure` block. 2017-07-05 10:21:42 +09:00
Robin Ward 0ba39109a0 FIX: Make `omit_default_port` the global default 2017-06-30 12:43:26 -04:00
Robin Ward 46a3b30021 FIX: omit the default port in the `Host` header 2017-06-30 12:36:02 -04:00
Sam 5c89a37ceb FIX: log when push notifications fail 2017-06-30 10:45:53 -04:00
Arpit Jalan 16d356ab4e FEATURE: resending invite should include original custom message
https://meta.discourse.org/t/will-resent-invite-include-original-custom-message/64699
2017-06-30 18:13:33 +05:30
Sam c1580b9d36 attempt to fix push notifications 2017-06-30 08:40:43 -04:00
Guo Xiang Tan 1ab60d83df PERF: Terminate scheduled job earlier if badge is not enabled. 2017-06-26 10:13:58 +09:00
Guo Xiang Tan 6cc174ca82 PERF: Fetch users in batches in grant anniversary badge job. 2017-06-23 16:54:35 +09:00
Régis Hanol 2e7753c27f User 'FileHelper.is_image?' to check wether a link is poiting to an image 2017-06-22 12:54:42 +02:00
Guo Xiang Tan 80e348d226 PERF: Speed up slow tests in our test suite.
Before

```
Finished in 7 minutes 23 seconds (files took 4.15 seconds to load)
7145 examples, 0 failures, 10 pending
```

After

```
Finished in 6 minutes 12 seconds (files took 4.41 seconds to load)
7145 examples, 0 failures, 10 pending
```
2017-06-22 11:23:31 +09:00
Guo Xiang Tan d82dbd565b FIX: Remove dependency on `rest-client`. 2017-06-16 09:42:41 +09:00
Régis Hanol d6c63cc5b2 FIX: user's default group should only be set once
Setting a user's default groups based on their email address should only be done once, ie. when they confirm their email address.
Previously we were doing this everytime we'd save a user record 🤷
2017-06-14 19:20:18 +02:00
Régis Hanol 5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
Guo Xiang Tan e888369f51 UX: Don't send emails for discobot notifications. 2017-06-12 17:00:27 +09:00
Robin Ward d92c938491 FIX: Don't raise an error cleaning up uploads if a path is nil 2017-06-08 14:31:58 -04:00
Régis Hanol e8f0771dc9 FIX: the 'clean_up_uploads' jobs would delete images used in site settings
when they were entered using absolute URLs, with the CDN or simple a different format than the one used in the database
2017-06-07 22:53:15 +02:00
Vinoth Kannan 209383faab FEATURE: pull onebox images 2017-06-06 18:51:58 +05:30
Sam 0aed2533ac Revert unread optimisation, has too many edge cases 2017-05-26 09:04:13 -04:00
Sam 29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Robin Ward cdbe027c1c Refactor `FileHelper` to use keyword arguments. 2017-05-24 13:54:26 -04:00
Robin Ward b78b010757 FIX: Fixes to grant new user of the month badge 2017-05-24 11:36:24 -04:00
Neil Lalonde 9edc490d3f FIX: remove memoized values in jobs 2017-05-22 16:26:30 -04:00
Robin Ward b23fc2bf84 Helper to find the final destination for a URL 2017-05-22 15:52:41 -04:00
Neil Lalonde f350dd4fd1 FIX: possible data leaking from one site to another in multisite in PendingFlagsReminder 2017-05-22 15:18:13 -04:00
Robin Ward 773445b8df FIX: Topic Crawling should only crawl HTTP/S urls 2017-05-22 11:57:20 -04:00
Guo Xiang Tan 4382a0bb07 Rename `PostTimestampChanger` -> `TopicTimestampChanger`. 2017-05-22 15:01:33 +08:00
Robin Ward ea9f93dcc5 FIX: Don't crawl non-http/s links 2017-05-19 16:57:41 -04:00
Neil Lalonde 7821400141 FEATURE: staff can set a timer to remind them about a topic 2017-05-16 14:49:50 -04:00
Robin Ward bd0bb879e5 FIX: Don't count likes from system users in the new user of the month 2017-05-15 13:14:32 -04:00
Guo Xiang Tan 8eecd42856 FIX: Delete topic timer after completion. 2017-05-12 10:28:51 +08:00
Neil Lalonde 55b61e9bea rename topic_status_update to topic_timer 2017-05-11 18:27:53 -04:00
Neil Lalonde 1019bbda46 FEATURE: set a timer to delete a topic 2017-05-11 12:52:29 -04:00
Robin Ward 3ade46de0b FIX: Don't send system message for new user of the month if disabled 2017-05-11 11:33:16 -04:00
Régis Hanol 4b7be137cb don't generate errors if 'upload_id' is invalid 2017-05-11 09:07:04 +02:00
Régis Hanol 9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Sam f709899a1d Allow theme field object model to support uploads 2017-05-10 15:46:42 -04:00
Robin Ward 4db76796b9 FEATURE: Setting to poll feeds more frequently 2017-05-10 14:30:12 -04:00
Robin Ward 6e4ba8a33e Catch RSS Parsing errors 2017-05-09 15:07:06 -04:00
Robin Ward c2bf525a06 Make CloseTopic a no-op just in case 2017-05-09 15:02:05 -04:00
Robin Ward ba1868b051 FIX: There are still jobs enqueued for CloseTopic 2017-05-09 14:58:37 -04:00
Arpit Jalan e6e0025326 FIX: handle BOM in bulk import CSV file 2017-05-09 22:38:07 +05:30
Arpit Jalan 94683b33b8 FIX: sanitize bulk invite error log 2017-05-09 17:41:59 +05:30
Robin Ward 4f6e5fed2a We don't need to raise an error when no post is present. Just noop. 2017-05-08 15:08:29 -04:00
Guo Xiang Tan 71a266b673 Remove daily mailing mode option as it doesn't scale.
https://meta.discourse.org/t/daily-updates-option-for-mailing-list-mode/45029/14?u=tgxworld
2017-05-05 12:21:50 +08:00
Guo Xiang Tan 50c8ae2c10 Load user records in batches. 2017-05-05 10:57:31 +08:00
Arpit Jalan 86f1cc8c92 FIX: don't apply max_emails_per_day_per_user on critical emails 2017-05-03 17:07:39 +05:30
Arpit Jalan cdce060a38 FIX: don't apply max emails per day per user to forgot password 2017-05-03 14:02:37 +05:30
Neil Lalonde ca224ed8e9 FIX: pending flags message should go to moderators group, not staff 2017-05-01 14:53:16 -04:00
Robin Ward dfe1174137 FIX: The dates for retroactive anniversary badges were wrong 2017-05-01 12:06:18 -04:00
Robin Ward f05f1a24d3 Change Anniversary badge to be multiple grant, once per year 2017-04-28 14:22:54 -04:00
Guo Xiang Tan 85d9342ff0 Merge pull request #4836 from tgxworld/fix_clean_up_staged_users_job
FIX: Clean up unused staged users job not completing.
2017-04-27 15:41:46 +08:00
Robin Ward bf9c4a7828 FEATURE: secure_email site setting to prevent data going out in email 2017-04-26 13:05:56 -04:00
Guo Xiang Tan fe0a7d97ca Fix incorrect class name. 2017-04-26 18:12:51 +08:00
Guo Xiang Tan b00886f5c1 Remove sidekiq-statistic gem.
* We don't really use it and there is an on-going issue
  with the gem not expiring keys in a Redis list which
  hasn't been fully resolved.

  https://github.com/davydovanton/sidekiq-statistic/issues/73
2017-04-26 14:54:47 +08:00
Guo Xiang Tan daa97c40ce FIX: Clean up unused staged users job not completing. 2017-04-26 10:51:36 +08:00
Guo Xiang Tan 423f2ab228 FIX: Processing incoming email should be done in a background job. 2017-04-24 13:57:28 +08:00
Neil Lalonde 05efa7ce68 simplify last_notified_id 2017-04-20 15:26:28 -04:00
Neil Lalonde 3835e16cf7 FIX: New implementation of the "notify about flag after" setting. Only notify about new flags since the last notification. Send a private message to staff. Mention the 3 most active moderators in the message so they get notification emails. 2017-04-19 16:17:45 -04:00
Guo Xiang Tan 73180c8a19 FIX: Private message can be set to publish in the future. 2017-04-11 20:44:25 +08:00
Guo Xiang Tan aadf4805a5 FIX: Topic status update not being deleted once it has been executed. 2017-04-11 10:32:13 +08:00
Régis Hanol db77640634 FIX: grant first reply by email job was *brokenated* 2017-04-10 22:11:29 +02:00
Guo Xiang Tan 690d2f4bd3 UX: Publish topic changes when topic is published. 2017-04-07 15:32:00 +08:00
Guo Xiang Tan 48a9860f07 FIX: Publish topic to a category should unlist it as well. 2017-04-07 13:58:52 +08:00
Guo Xiang Tan 8e794070d2 Fix incorrect method name. 2017-04-06 13:45:59 +08:00
Guo Xiang Tan e0e06c2992 Merge pull request #4794 from tgxworld/nuke_old_auto_close_jobs
FIX: Add onceoff job to remove old auto close topic sidekiq jobs.
2017-04-06 12:29:42 +08:00
Guo Xiang Tan 5943543ec3 FIX: Improve checks for non-human users. 2017-04-06 11:29:34 +08:00
Guo Xiang Tan e8eaffbd72 FIX: Add onceoff job to remove old auto close topic sidekiq jobs. 2017-04-06 10:52:13 +08:00
Robin Ward 17f2974d0a SECURITY: Confirm new administrator accounts via email 2017-04-04 15:59:01 -04:00
Guo Xiang Tan 69661be500 FIX: Don't raise an error if `TopicStatusUpdate` has been deleted. 2017-04-04 14:29:47 +08:00
Guo Xiang Tan ad44d2a400 Merge pull request #4790 from tgxworld/add_publish_to_topic_status_update
FEATURE: Allow admins to schedule a topic to be published in the future.
2017-04-04 11:18:53 +08:00
Guo Xiang Tan f4758a4c4d FEATURE: Allow admins to schedule a topic to be published in the future. 2017-04-04 11:16:05 +08:00
Robin Ward ebe232da47 Weigh staff votes more for New User of the Month badges 2017-04-03 16:52:31 -04:00
Robin Ward a13a8dc96c FIX: Safeguard to not award NewUserOfTheMonth if it hasn't been a month 2017-04-03 13:26:51 -04:00
Robin Ward 3d64f33d40 Require at least two likes to get the "New User of the Month" badge 2017-04-03 13:03:28 -04:00
Jeff Atwood 3b64aeaf21 loosen new user of the month criteria slightly 2017-03-31 16:33:50 -07:00
Robin Ward e4b1376146 Change New User of the Month Badge to use trust level rather than age 2017-03-31 16:56:58 -04:00
Robin Ward a8d3779f0b Update New User of the Month badge to require 4 posts in 2 topics 2017-03-31 16:52:00 -04:00
Robin Ward fc7fa4c0ad Rename "Rookie of the Month" to "New User of the Month" 2017-03-31 16:30:30 -04:00
Robin Ward e6f6bd34d8 Send the user a notice when they've received rookie of the month 2017-03-31 15:06:31 -04:00
Robin Ward 893e93dfbe New badge, Rookie of the Month, for two new high quality users. 2017-03-31 15:06:31 -04:00
Guo Xiang Tan 34b7bee568 FEATURE: Allow admin to auto reopen at topic.
* This commit also introduces a `TopicStatusUpdate`
  model to support other forms of deferred topic
  status update in the future.
2017-03-31 11:14:18 +08:00
Arpit Jalan df246c79b6 FEATURE: add user profile fields in user list export 2017-03-21 15:51:52 +05:30
Guo Xiang Tan 566f367fc3 FIX: Don't migrate custom emojis that are no longer valid.
* Warn about failed migration in logs.
2017-03-17 08:28:24 +08:00