Commit Graph

249 Commits

Author SHA1 Message Date
Sam 0aed2533ac Revert unread optimisation, has too many edge cases 2017-05-26 09:04:13 -04:00
Sam 29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Neil Lalonde 7821400141 FEATURE: staff can set a timer to remind them about a topic 2017-05-16 14:49:50 -04:00
Neil Lalonde 55b61e9bea rename topic_status_update to topic_timer 2017-05-11 18:27:53 -04:00
cpradio 20c2c66dd4 FEATURE: Add normal as a preference for topic subscription state when replying to a topic 2017-04-20 22:33:10 -04:00
Guo Xiang Tan e4d7e4fbe3 Improve error messages. 2017-04-04 17:19:14 +08:00
Guo Xiang Tan 34b7bee568 FEATURE: Allow admin to auto reopen at topic.
* This commit also introduces a `TopicStatusUpdate`
  model to support other forms of deferred topic
  status update in the future.
2017-03-31 11:14:18 +08:00
Robin Ward cf7774bdd9 FEATURE: Block muted users from sending you PMs 2017-01-03 14:51:53 -05:00
Sam 2b808ad9da Merge pull request #4609 from joebuhlig/category-topics-wiki
FEATURE: Category setting to make all topics wikis
2016-12-20 09:15:51 +11:00
Neil Lalonde 923cf73c6e Topic Featured Links: move data from custom fields to topics and categories tables. Invert behaviour of topic_featured_link_allowed checkbox. Fix a bug with invalid topic records due to changing that category checkbox. 2016-12-19 14:54:07 -05:00
Joe Buhlig 87251fded7 FEATURE: Category setting to make all topics wikis
FEATURE: Category setting to make all topics wikis
2016-12-19 06:42:18 -06:00
Neil Lalonde a4c4f13901 Remove the topic_featured_link_onebox setting. We will always try to onebox a link and add it to the body if topic_featured_link_enabled is enabled. 2016-12-09 13:28:12 -05:00
Erick Guan 52763f5115
FEATURE: Allow posting a link with topics 2016-12-05 17:20:54 +01:00
Sam c04d4171ff FIX: whisper no longer experimental
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
2016-12-02 17:03:31 +11:00
cpradio 6f1c31d777 Add notification level user preference when replying to a topic 2016-09-30 14:58:07 -04:00
Robin Ward 79245a25a3 Explain the `skip_jobs` parameter 2016-08-24 11:06:14 -04:00
Robin Ward 2cb4cb7b72 FIX: Don't enqueue jobs in a transaction 2016-08-23 16:07:59 -04:00
Guo Xiang Tan 5fe4837e28 Add `PostCreator#create!`. 2016-07-15 11:36:06 +08:00
Robin Ward baae2a4b7c FIX: Don't update the last read post when disable jump reply is set 2016-06-08 14:08:19 -04:00
Neil Lalonde f13470b96b Use db schema for tags instead of plugin store and custom fields 2016-05-26 14:29:48 -04:00
Neil Lalonde 0101528821 PERF: update_user_counts is too slow during imports 2016-05-17 13:22:32 -04:00
Neil Lalonde 3deda3ca39 skip auto-close during import 2016-05-17 11:16:51 -04:00
Neil Lalonde 1625b08c86 FIX: posting my fail because there are no tags 2016-05-06 11:20:17 -04:00
Arpit Jalan 53809d526c FIX: show proper error message when no user selected for private message 2016-05-05 18:47:08 +05:30
Neil Lalonde e5918c7d00 FEATURE: Merge tagging plugin into core 2016-04-27 11:58:53 -04:00
Régis Hanol 2194ccec6e slightly better automatic flag reason messages 2016-04-25 23:03:17 +02:00
Jeff Atwood f39186b24a FIX: autoclose topic max replies used wrong constant 2016-04-12 13:42:52 -07:00
Sam 0113fce420 FEATURE: automatically close topics with 10k+ posts
FEATURE: automatically close messages with 2k+ posts

Both configurable via `auto_close_messages_post_count`
and `auto_close_topics_post_count`
2016-04-12 13:29:48 +10:00
Sam Saffron b0567f9c62 FEATURE: automatically sync "move to inbox" / "archive" state on messages 2016-02-07 23:39:07 +11:00
Sam d0ee32f3ce FIX: correct counts on user summary 2016-01-24 16:39:01 +11:00
Sam 1a27d887fa FIX: if topic did not save do not attempt to skip inbox 2016-01-14 10:38:20 +11:00
Sam 4ec409f705 FEATURE: don't add admins to topic allowed users if already in group 2016-01-12 13:57:45 +11:00
Régis Hanol cf4cb2126a FIX: word_count wasn't working with non-latin sentences 2016-01-11 11:16:23 +01:00
Sam 03ea0bfe22 FEATURE: allow users to archive messages
Messages are now in 3 buckets

- Inbox for all new messages
- Sent for all sent messages
- Archive for all messages you are done with

You can select messages from your Inbox or Sent and move them to your Archive,
you can move messages out of your Archive similarly

Similar concept applied to group messages, except that archiving and unarchiving
will apply to all group members
2015-12-23 11:09:30 +11:00
Régis Hanol 31a54377be staged users automatically watches all topics they participates in 2015-11-18 22:24:46 +01:00
Sam 606aeb9d55 improvements to importer
- improve perf of test for existing posts
- always use a system guardian when importing posts
- for lithuim importer requery raw (transform is not repeatable)
2015-10-16 10:41:21 +11:00
Régis Hanol 54d6d24cbf FIX: timing issue with edits and cook post processing 2015-09-29 18:51:26 +02:00
Robin Ward 64598ceaba FIX: Replies to whispers *must* be whispers 2015-09-25 11:29:40 -04:00
Robin Ward 79beb9f409 FIX: You could set `reply_to_post_number` when replying as a new topic 2015-09-22 13:32:19 -04:00
Robin Ward 9f89aefdd3 FIX: Don't update the last poster when a whisper is made 2015-09-22 12:23:37 -04:00
Sam 5f5d055a86 only if \u0000 is included for the perf 2015-08-19 12:32:45 +10:00
Sam 431def1f52 need to dup strings, some may be frozen 2015-08-19 12:32:45 +10:00
Sam 714f841f0a FIX: null bytes in user input should not cause post creation to fail 2015-08-19 12:15:38 +10:00
Régis Hanol ac8e90674b FIX: be more lenient when creating a post with a username that is no longer valid 2015-08-12 15:54:51 +02:00
Sam 7b8b96446e FEATURE: track statistics around post creation
- how long were people typing?
- how long was composer open?
- how many drafts were created?
- correct, draft saved to go away after you continue typing

store in Post.find(xyz).post_stat
2015-08-03 14:29:15 +10:00
Robin Ward bb93a345eb UX: Use smaller messages for moderator actions. 2015-07-27 10:09:21 -04:00
Robin Ward fb8ba5e137 FIX: `PG::UniqueViolation` when trying to use the same embed code
Previously providing an embed code already in use would result in
a logged server error. After this commit the error is gracefully
bubbled up from the `PostCreator`
2015-06-15 12:09:59 -04:00
Robin Ward c6cd1928be Simple "cook" for email imports from mailing lists 2015-06-05 11:46:36 -04:00
Régis Hanol a737090442 - FEATURE: revamped poll plugin
- add User.staff scope
- inject MessageBus into Ember views (so it can be used by the poll plugin)
- REFACTOR: use more accurate is_first_post? method instead of post_number == 1
- FEATURE: add support for JSON-typed custom fields
- FEATURE: allow plugins to add validation
- FEATURE: add post_custom_fields to PostSerializer
- FEATURE: allow plugins to whitelist post_custom_fields
- FIX: don't bump when post did not save successfully
- FEATURE: polls are supported in any post
- FEATURE: allow for multiple polls in the same post
- FEATURE: multiple choice polls
- FEATURE: rating polls
- FEATURE: new dialect allowing users to preview polls in the composer
2015-04-23 19:33:29 +02:00
Robin Ward 2713b77e28 FIX: If validations were disabled some rate limits continued to run 2015-04-21 13:16:05 -04:00
Robin Ward 5990ab855b PERF: Move post alerting into async 2015-04-20 13:34:57 -04:00
Robin Ward 19a9a8b408 `NewPostManager` determines whether to queue a post or not 2015-04-15 14:54:36 -04:00
Robin Ward a5ee45ccbe `PostEnqueuer` object to handle validation of enqueued posts 2015-04-15 14:54:36 -04:00
Régis Hanol eec1921ba9 FIX: notifications inconsistencies + never notify system user 2015-04-01 11:55:59 +02:00
Sam 4720d0c12b Merge pull request #3277 from techAPJ/patch-1
FIX: return proper error message when replying to a closed or deleted topic
2015-03-17 07:38:00 +11:00
Arpit Jalan 484ae8b6b7 FIX: return proper error message when replying to a closed or deleted topic 2015-03-16 16:43:22 +05:30
Gerhard Schlager 53591ade00 Adds a few enhancements to the base importer
- Allows importing globally pinned topics
- Doesn't restore the original value of a SiteSetting anymore if an importer changed its value
- Allows all file extensions during the import
- Adds the ability to import bookmarks
2015-03-13 21:24:11 +01:00
Régis Hanol c8631a7a8b FIX: auto-message shouldn't generate notifications 2015-03-11 19:07:17 +01:00
Régis Hanol 6c4d852011 Improve vBulletin importer
- FEATURE: TopicCreator now supports 'pinned_at' parameter
- FIX: 🐛 FIX TopicQuerySQL to support pinned topic older than 2010
- FIX: 🐛 Properly remove all HTML Entities from Usernames/Titles/Category Names/Groups in vBulletin importer
- FIX: 🐛 Properly handle specific vBulletin BBCode (quotes/mentions)
- FIX: 🐛 Make sure we generate a username from the name of the user instead of a fake email
- FEATURE: Allow for custom timezone in vBulletin importer
- FEATURE: Support for profile pictures/background in vBulletin importer
- FIX: 🐛 merge the categories tree to only 2 levels in vBulletin importer
2015-01-19 15:00:55 +01:00
riking 399af512aa Update documentation for PostCreator 2015-01-05 07:52:33 -08:00
Sam 2251877332 FIX: "Dismiss Posts" corrupting read state
REFACTOR: seen_post_count was a bad name, renamed to highest_seen_post_number
2014-10-31 09:40:35 +11:00
Arpit Jalan 2131a37811 FEATURE: show raw email for replies/topics created via email 2014-10-17 22:22:19 +05:30
Régis Hanol 5754e8dd0f FEATURE: auto-close topics based on last post 2014-10-10 18:21:44 +02:00
Robin Ward 334e21a03a Revert "Revert "FEATURE: Can create warnings for users via PM""
This reverts commit 1c7559380c.
2014-09-08 11:11:56 -04:00
Robin Ward 1c7559380c Revert "FEATURE: Can create warnings for users via PM"
This reverts commit b0bfc1f93f.
2014-09-08 10:38:59 -04:00
Robin Ward b0bfc1f93f FEATURE: Can create warnings for users via PM 2014-09-08 10:27:06 -04:00
Robin Ward 1c5e8efa68 FEATURE: Show an email icon beside posts that arrived via email 2014-09-04 13:04:41 -04:00
Robin Ward ab02ef4221 Merge pull request #2718 from riking/plugin-events
FEATURE: Add a plugin callback when editing a post
2014-09-02 10:46:08 -04:00
Régis Hanol 61db6c95d2 FIX: couldn't start a 4-spaces block on the first line of a post 2014-09-02 01:18:06 +02:00
riking 085b18577c Remove unnecessary user/topic load in PostCreator 2014-08-28 22:07:40 -07:00
riking 24ba7ccfbb FEATURE: Add a plugin callback when editing a post
FIX: Wrap PostRevisor database work in a transaction
2014-08-28 21:27:40 -07:00
riking 3396e6fea3 Centralize MessageBus post updates
After this change, only two files directly publish to MessageBus with a
topic interpolated in the channel: Post and TopicUser.
2014-08-28 20:40:36 -07:00
Régis Hanol f4208ae83f FEATURE: normalize whitespaces in topic title/post content 2014-08-12 00:01:58 +02:00
Sam 138d013e56 FIX: Suspended users should not be allowed to post 2014-07-31 13:15:16 +10:00
Sam 226a197335 fix logic take2 2014-07-30 14:24:20 +10:00
Sam c5238026a9 fix logic 2014-07-30 14:18:01 +10:00
Sam 9147af1d62 FIX: eliminate race condition creating posts
FIX: correct message bus posting
2014-07-30 14:18:01 +10:00
Régis Hanol bddffa7f9a FEATURE: flag dispositions normalization
All flags should end up in one of the three dispositions
  - Agree
  - Disagree
  - Defer

In the administration area, the *active* flags section displays 4 buttons
  - Agree (hide post + send PM)
  - Disagree
  - Defer
  - Delete

Clicking "Delete" will open a modal that offer to
  - Delete Post & Defer Flags
  - Delete Post & Agree with Flags
  - Delete Spammer (if available)

When the flag has a list associated, the list will now display 1
response and 1 reply and a "show more..." link if there are more in the
conversation. Replying to the conversation will NOT give a disposition.
Moderators must click the buttons that does that.

If someone clicks one buttons, this will add a default moderator message
from that moderator saying what happened.

The *old* flags section now displays the proper dispositions and is
super duper fast (no more N+9999 queries).

FIX: the old list includes deleted topics
FIX: the lists now properly display the topic states (deleted, closed,
archived, hidden, PM)
FIX: flagging a topic that you've already flagged the first post
2014-07-28 19:28:07 +02:00
Sam 0f9678fe49 FIX: faster update of all badges
Introduced badge triggers, introduced concept of badge that happens due to a post but has the post hidden

Delta badge grant happens once a minute, backed by redis
2014-07-23 11:46:07 +10:00
Sam 2d0def9940 FIX: First Quote badge bust
Feature: track quoted posts
2014-07-15 17:47:24 +10:00
Neil Lalonde 7d5d5862c1 Import optimizations for topic creation: Prevent queuing of most jobs when importing posts and topics. Only do some recalculations at the end of the import. 2014-07-03 16:24:47 -04:00
Sam c6c412fd45 BUGFIX: no reading credit for posts you create 2014-06-04 14:10:54 +10:00
Sam 4f7e960e49 raw bbpress import script
amend post_creator to support custom fields
2014-05-18 11:33:59 +10:00
Louis Rose 1574485443 Perform the where(...).first to find_by(...) refactoring.
This refactoring was automated using the command: bundle exec "ruby refactorings/where_dot_first_to_find_by/app.rb"
2014-05-06 14:41:59 +01:00
Robin Ward dbab628e16 Support for creating embedded topics via API 2014-04-03 14:42:44 -04:00
Sam 090f5c99c2 FEATURE: Live update edits 2014-03-24 12:19:08 +11:00
Robin Ward d3f1eb395d Updated import for TypePad 2014-03-19 15:02:49 -04:00
Neil Lalonde 0b1550f9d4 Add excerpt column to topics table to remove N+1 query in ListableTopicSerializer 2014-03-18 15:24:29 -04:00
Sam 5c26b3dad1 FIX: broken specs after new link alerting code 2014-03-18 15:22:53 +11:00
Sam 24667cedee FEATURE: notify users when linked
Extract out PostAlerter from observer
Track outgoing post links correctly
Notify users they are linked when linked
2014-03-18 15:22:53 +11:00
Wojciech Zawistowski b097cf4997 Refactors PostCreator. 2014-02-17 01:57:37 -05:00
Wojciech Zawistowski 2a7b609cae Refactors PostCreator. 2014-02-10 20:29:31 +01:00
Sam 227873df78 FEATURE: proper mailing list mode
once enable_mailing_list_mode is enabled any user can elect
to get every post via email unless they opt out of category or topic
2014-02-07 11:07:52 +11:00
Robin Ward 4f8aed295a FEATURE: Embeddable Discourse comments, now with simple-rss instead of feedzirra 2013-12-31 15:01:22 -05:00
Robin Ward 62db063e1e Revert "Support for Embeddable Comments via IFRAME" - it depends on Curl
which not every server has. Have to rethink this.

This reverts commit e3e4c62887.
2013-12-31 12:52:31 -05:00
Robin Ward e3e4c62887 Support for Embeddable Comments via IFRAME 2013-12-31 12:26:24 -05:00
Robin Ward 6853f37bee Show estimated reading time near summarize button. 2013-12-10 13:47:36 -05:00
Sam 5e0b22365a skip validations for Spam Rules when needed 2013-12-02 16:13:47 +11:00
Sam ff957a7b37 rails 4 deprecation fixes 2013-10-24 13:02:25 +11:00
Sam 666264879c change it so all topics MUST include a category, we store a special uncategorized category to compensate
this cleans up a bunch of internals and removes some settings
2013-10-24 12:08:02 +11:00
Brian Alexander e2a5415aca fix #1567; nil reference in spam_rules_enforcer.rb 2013-10-22 19:14:31 -06:00
Sam 1ee49798b2 work in progress wide category list 2013-10-18 11:10:10 +11:00
Neil Lalonde 3c2c6ab24b Add sockpuppet spammer detection. Automatically flag posts if they are from new users (registered less than 24 hours ago) at the same IP address and one of them started the topic. 2013-10-11 13:34:05 -04:00
Sam 5bf26ec34e large refactor, ship a few columns from the user table into user_stats 2013-10-07 15:04:59 +11:00
Thomas McDonald 5ad8718a87 Remove duplicate create method in PostCreator 2013-09-21 22:02:51 +01:00
Neil Lalonde 827680766e Show which domains were used in the spam_post_blocked notification 2013-09-12 17:33:30 -04:00
Neil Lalonde 78c15d5810 Move the unique post key storage code into the Post model 2013-09-09 16:17:31 -04:00
Sam 41a1b6942d notify moderators now goes to the "community" user, that saves our poor mods from a flood of pms
if any staff respond to a pm they are automatically added to the list of recipients and will start
getting email notifications
2013-09-06 14:07:23 +10:00
Robin Ward fe3a69c271 Private message emails now include the history 2013-08-07 13:03:45 -04:00
Sam 58e7c3e1f3 push some work out of AR callbacks into PostCreator, add a couple of helpers for post and topic creation in test
fix it so the auto_track false marks topics as new
2013-07-22 15:07:20 +10:00
Sam 31bb08bcdd added an option to bypass auto tracking of topics on post creation 2013-07-22 15:07:20 +10:00
Sam ecf17cfebb work in progress, add fidelity to category group permissions (full, create posts, readonly) 2013-07-16 15:46:11 +10:00
Sam f6a8f6cfe5 don't notify on moderator actions, they just cause noise,
also filter out pms properly while at it
2013-07-08 12:44:55 +10:00
Sam f6b850e7a4 allow skipping the validations on creation if its an api call AND skip_validations is specified
this allows wordpress plugin to post very very short titles or titles that would otherwise be disallowed
2013-07-02 12:23:19 +10:00
Robin Ward 023d18cab2 FIX: Admins should be able to post short titles. This is especially important for when syndicating
a blog via the wordpress plugin into Discourse.
2013-06-29 17:57:10 -04:00
Sam 966513a66d my spec suite has been playing up attempting to clean some stuff by avoiding type checks 2013-06-28 12:18:04 +10:00
Robin Ward bb908d5913 Email parsing uses Traditional Markdown Linebreaks by default. Added JS tests for line breaks. 2013-06-21 11:37:26 -04:00
Neil Lalonde 9b1d0baf45 Send a message to moderators when a newuser_spam_host_threshold is exceeded. Send it no more than once per day per user. 2013-06-10 13:17:32 -04:00
Navin 3fdba0019b Extract callbacks and validations for Post
Move Post create callbacks to PostCreate
Extract Post validations
Move stripped_length_validator to lib/validators
2013-06-09 20:47:04 +02:00
Robin Ward 1b5ec42173 FIX: Regression where a topic rollback would not properly report errors to the client. 2013-06-07 12:36:37 -04:00
Ian Christian Myers b61e10f9ad All parameters for #create in PostsController pass through strong_parameters.
We are now explicitly whitelisting all parameters for Post creation. A nice side-effect is that it cleans up the #create action in PostsController. We can now trust that all parameters entering PostCreator are of a safe scalar type.
2013-06-07 01:29:25 -07:00
Navin a4b73b2473 Simplify PostCreator - Introduce TopicCreator
Extract code to smaller methods
2013-06-04 20:13:01 +02:00
Sam e93b7a3b20 more progress towards live unread and new counts, unread message implemented, still to implement delete messages 2013-05-30 16:49:57 +10:00
Sam 73834370a5 work in progress, live unread and new counts 2013-05-30 16:49:57 +10:00
Sam d2f2a8e218 refactoring 2013-05-28 09:13:53 +10:00
Sam e91ed83586 the private message stream is different to normal streams, improving the ui a bit and collapsing conversations 2013-05-20 16:44:06 +10:00
Sander Datema 286a06bd4e Set created_at manually through PostCreator 2013-05-18 21:24:29 +02:00
Robin Ward d554a59102 Support for a new site setting: `newuser_spam_host_threshold`. If a new user posts a link
to the same host enough tiles, they will not be able to post the same link again.

Additionally, the site will flag all their previous posts with links as spam and they will
be instantly hidden via the auto hide workflow.
2013-05-16 12:19:50 -04:00
Sam e9ebadb414 Nuke message_bus_observer move to service class and classes
Secure all messages triggered by post creation and all user actions so they don't leak
(meaning, if you have a browser open and secure topics are created you will only get them if you are allowed to see them)
2013-05-16 15:03:16 +10:00
Robin Ward b1bfda0e11 Don't show new user education when editing a reply. 2013-05-15 15:16:42 -04:00
Robin Ward be234ce9b9 You can now 'move' the first post of a topic to another topic as a merge. In that
case, the first post is cloned instead of being deleted from the original topic.
2013-05-13 14:07:31 -04:00
Neil Lalonde 9828c87525 Topic Auto-Close: admins and mods can set a topic to automatically close after a number of days 2013-05-13 12:53:52 -04:00
Sam 9b33e826f2 clear flags on reply to notify moderators 2013-05-13 11:48:01 +10:00
Sam 5280b3a01b more group progress, UI getting there, controller mostly done
changed it so notify moderators goes to the moderators group
allow admins to grant self moderation and revoke self moderation
2013-05-09 17:37:34 +10:00
Sam 65cd00cf25 moderators now have teeth, more at http://meta.discourse.org/t/moderator-permission-set/6307/5
allow pms to be targetted at groups
2013-05-02 15:15:53 +10:00
Neil Lalonde 3b6aeb14c7 Add subtype to topics to classify private messages 2013-04-16 16:56:18 -04:00
Robin Ward 8942d9a094 Okay, THIS time the specs should pass. Geez. 2013-04-08 11:23:03 -04:00
Robin Ward 9c461aced4 Safer code - check that topic is present. 2013-04-08 11:01:55 -04:00
Robin Ward 1715aa66d9 Fixes issue with featuring topic users 2013-04-08 10:51:01 -04:00
Sam a7f92ec0c1 topic reply count resync for each new post 2013-04-05 17:47:54 +11:00
Sam 4fbf017272 get regular trust level going, self heal inconsistent topic timings 2013-04-05 15:30:28 +11:00
Robin Ward a819a26f34 BUGFIX: New status posts weren't using PostCreator 2013-03-28 16:40:54 -04:00
Karan Misra 5dfb04e4b3 Convert a lot of :a => b to a: b and bring peace to the world 2013-03-25 05:07:36 +05:30
Robin Ward 897d48b145 Moved more `Post` callbacks into service classes 2013-03-18 17:54:55 -04:00
Robin Ward d5e4243f02 Move draft sequence incrementing out of `after_save` 2013-03-18 17:54:50 -04:00
Robin Ward aea848f164 Move MessageBus observer into PostCreator 2013-03-18 17:54:39 -04:00
Robin Ward b6224b014c Move a bunch of callbacks into PostCreator 2013-03-18 13:56:26 -04:00
Gosha Arinich cafc75b238 remove trailing whitespaces ❤️ 2013-02-26 07:31:35 +03:00
Robin Ward 21b5628528 Initial release of Discourse 2013-02-05 14:16:51 -05:00