discourse/lib
David Taylor bf6f8299a7 FEATURE: Pull hotlinked images immediately after posting
Previously, with the default `editing_grace_period`, hotlinked images were pulled 5 minutes after a post is created. This delay was added to reduce the chance of automated edits clashing with user edits.

This commit refactors things so that we can pull hotlinked images immediately. URLs are immediately updated in the post's `cooked` HTML. The post's raw markdown is updated later, after the `editing_grace_period`.

This involves a number of behind-the-scenes changes including:

- Schedule Jobs::PullHotlinkedImages immediately after Jobs::ProcessPost. Move scheduling to after the `update_column` call to avoid race conditions

- Move raw changes into a separate job, which is delayed until after the ninja-edit window

- Move disable_if_low_on_disk_space logic into the `pull_hotlinked_images` job

- Move raw-parsing/replacing logic into `InlineUpload` so it can be easily be shared between `UpdateHotlinkedRaw` and `PullUserProfileHotlinkedImages`
2022-05-23 14:28:02 +01:00
..
action_dispatch/session DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
auth FEATURE: Add fallback to suggested value when auth_overrides_username (#16612) 2022-05-04 11:22:09 +01:00
autospec DEV: Support for running theme test with Ember CLI (third attempt) 2022-01-13 16:02:07 -05:00
backup_restore DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
common_passwords DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
compression
content_security_policy DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
demon DEV: Route Sidekiq logs to Rails logger (#15817) 2022-02-04 16:28:20 +00:00
discourse_dev DEV: Add ability to populate private messages (#16705) 2022-05-12 14:29:49 -04:00
email FIX: Use our header value instead of custom header on duplicates (#16711) 2022-05-11 13:47:12 +10:00
emoji FEATURE: Add missing emojis (#15582) 2022-01-14 17:51:13 -03:00
faker DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
file_store DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
freedom_patches DEV: Upgrade to Rails 7.0.3 2022-05-10 12:01:00 +02:00
generators/rails DEV: removes plugin generator (#14101) 2021-08-20 11:29:06 +02:00
guardian FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
i18n DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
imap FEATURE: Introduce site setting to allow for non staff pm tagging (#16671) 2022-05-10 10:02:28 -05:00
import
import_export FEATURE: include user custom fields in base exporter (#14690) 2021-10-22 10:02:56 -07:00
javascripts DEV: Disallow `Ember` global usage (#16147) 2022-03-09 17:54:07 +01:00
middleware DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
migration DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
onebox DEV: Don’t patch Sanitize::Config 2022-04-06 17:10:51 +02:00
plugin DEV: Add discourse-gamification to the official plugins (#16639) 2022-05-04 21:06:06 +02:00
pretty_text DEV: replaces huge generated emoji list by a simpler regex (#11053) 2021-04-22 08:43:06 +02:00
rate_limiter FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
reviewable DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
scheduler
search DEV: Remove monkey patch that is no longer required (#16648) 2022-05-06 10:33:42 +08:00
second_factor FEATURE: Add 2FA support to the Discourse Connect Provider protocol (#16386) 2022-04-13 15:04:09 +03:00
seed_data FIX: Support Ruby 3 keyword arguments 2021-10-05 11:25:00 -04:00
sidekiq
site_settings FEATURE: Validate setting combination between exif strip and img opt (#16662) 2022-05-05 15:13:17 -03:00
stylesheet DEV: Fix stylesheet watcher (#16635) 2022-05-04 11:58:52 -04:00
tasks DEV: Only demux migration stdout if running concurrently (#16895) 2022-05-23 14:26:13 +01:00
theme_store DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
topic_query FIX: exclude topics from muted tag in category featured list. (#14925) 2021-11-16 12:10:50 +05:30
turbo_tests FIX: Make thumbnail tests start with a clean slate (#15216) 2021-12-07 13:07:45 -06:00
validators FEATURE: Allow category moderators to post consecutively (#16706) 2022-05-10 22:18:17 +03:00
webauthn
wizard FEATURE: Enable auto dark mode on new instances (#14208) 2021-09-02 14:55:38 -04:00
admin_confirmation.rb FEATURE: Centralized 2FA page (#15377) 2022-02-17 12:12:59 +03:00
admin_constraint.rb Revert "DEV: Add context in `AdminConstraint` (#15838)" (#15845) 2022-02-07 21:05:19 +03:00
admin_user_index_query.rb FIX: Add email to admin user list when show_emails is enabled (#16636) 2022-05-04 14:07:22 -03:00
age_words.rb
archetype.rb
auth.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
backup_restore.rb FIX: Handle enum types during database restore (#16624) 2022-05-04 08:40:34 +10:00
badge_posts_view_manager.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
badge_queries.rb FIX: Don't grant sharing badges to users who don't exist (#13851) 2021-07-27 16:32:59 +10:00
base62.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
bookmark_manager.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
bookmark_query.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
bookmark_reminder_notification_handler.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
browser_detection.rb
cache.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
canonical_url.rb FEATURE: Send a 'noindex' header in non-canonical responses (#15026) 2021-11-25 16:58:39 -03:00
category_badge.rb
chrome_installed_checker.rb DEV: Add chromium to ChromeInstalledChecker (#16224) 2022-03-19 11:00:06 +01:00
comment_migration.rb
common_passwords.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
composer_messages_finder.rb FEATURE: Make allow_uploaded_avatars accept TL (#14091) 2021-08-24 10:46:28 +03:00
configurable_urls.rb Replace `base_uri` with `base_path` (#10879) 2020-10-09 12:51:24 +01:00
content_buffer.rb
content_security_policy.rb PERF: Eager load Theme associations in Stylesheet Manager. 2021-06-21 11:06:58 +08:00
cooked_post_processor.rb FEATURE: Pull hotlinked images immediately after posting 2022-05-23 14:28:02 +01:00
cooked_processor_mixin.rb DEV: Map already-downloaded hotlinked images in post_process_cooked 2022-05-23 14:28:02 +01:00
crawler_detection.rb FEATURE: Implement browser update in crawler view (#12448) 2021-03-22 19:41:42 +02:00
csrf_token_verifier.rb
current_user.rb
custom_renderer.rb
db_helper.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
directory_helper.rb
discourse.rb DEV: New readonly mode. Only applies to non-staff (#16243) 2022-05-17 13:06:08 -05:00
discourse_connect_base.rb FEATURE: Add 2FA support to the Discourse Connect Provider protocol (#16386) 2022-04-13 15:04:09 +03:00
discourse_connect_provider.rb FEATURE: Add 2FA support to the Discourse Connect Provider protocol (#16386) 2022-04-13 15:04:09 +03:00
discourse_dev.rb DEV: move `discourse_dev` gem to the core. (#13360) 2021-06-14 20:34:44 +05:30
discourse_diff.rb Escape values of HTML attributes 2021-08-10 10:25:15 -04:00
discourse_event.rb DEV: Remove site_setting_saved event (#15164) 2021-12-02 09:33:03 -06:00
discourse_hub.rb
discourse_ip_info.rb DEV: Typo. (#16092) 2022-03-03 09:24:58 +08:00
discourse_js_processor.rb DEV: Allow Ember CLI assets to be used by development Rails app (#16511) 2022-04-21 16:26:34 +01:00
discourse_logstash_logger.rb FIX: Use 'hostname' when Discourse.os_hostname is not available 2020-02-18 13:37:39 +02:00
discourse_plugin_registry.rb REFACTOR: Improve support for consolidating notifications. (#14904) 2021-11-30 13:36:14 -03:00
discourse_redis.rb DEV: Readonly Redis support for `DiscourseRedis#multi/pipelined` (#16744) 2022-05-13 16:18:13 +08:00
discourse_sourcemapping_url_processor.rb DEV: Fix production sourcemaps with Ember CLI (#16707) 2022-05-11 10:23:32 +01:00
discourse_tagging.rb FEATURE: Show prompt for required tag groups (#16458) 2022-04-21 13:13:52 +01:00
discourse_updates.rb FIX: Regression introduced in #14715 (#14842) 2021-11-09 17:20:09 +11:00
disk_space.rb FIX: correct upload statistics report for external storage 2020-02-20 15:15:53 +11:00
distributed_cache.rb FIX: Handle `nil` values in `DistributedCache#defer_get_set` (#15978) 2022-02-17 14:52:14 +00:00
distributed_memoizer.rb DEV: Make DistributedMemoizer use DistributedMutex (#16229) 2022-04-05 19:29:58 +02:00
distributed_mutex.rb FIX: add support for pipelined and multi redis commands (#16682) 2022-05-10 08:19:02 +10:00
edit_rate_limiter.rb FEATURE: Increase daily edit limits proportionally to trust level (#13090) 2021-05-19 13:57:21 +04:00
email.rb DEV: pull email address validation out to a new EmailAddressValidator 2022-02-17 21:49:22 -05:00
email_backup_token.rb
email_cook.rb PERF: Avoid lookbehinds when replacing links in imported emails (#11931) 2021-02-02 17:34:00 +01:00
email_updater.rb DEV: Update :critical_user_email calls to use strings (#15827) 2022-02-04 23:43:53 +00:00
ember_cli.rb DEV: Fix production sourcemaps with Ember CLI (#16707) 2022-05-11 10:23:32 +01:00
encodings.rb
enum.rb
enum_site_setting.rb
excerpt_parser.rb DEV: Remove dead code 2021-05-31 10:22:50 +08:00
external_upload_helpers.rb FIX: Use hidden site setting for batch presign rate limit (#16692) 2022-05-10 11:14:26 +10:00
feed_element_installer.rb
feed_item_accessor.rb FIX: Select best link from Atom feed (#15663) 2022-01-21 17:54:18 +02:00
file_helper.rb DEV: Expand UploadMarkdown generation capabilities (#15930) 2022-02-14 15:48:27 +10:00
filter_best_posts.rb
final_destination.rb FEATURE: Site setting for blocking onebox of URLs that redirect (#16881) 2022-05-23 13:52:06 +03:00
flag_query.rb DEV: Remove deprecated methods (#14885) 2021-11-11 12:21:25 -06:00
flag_settings.rb DEV: Compatibility with TruffleRuby (#16864) 2022-05-19 21:56:55 +02:00
gaps.rb
global_path.rb FEATURE: Centralized 2FA page (#15377) 2022-02-17 12:12:59 +03:00
group_email_credentials_check.rb FEATURE: Scheduled group email credential problem check (#15396) 2022-01-04 10:14:33 +10:00
guardian.rb FIX: staff should not be able to PM groups that "Nobody" can message (#16163) 2022-03-22 10:23:14 +10:00
has_errors.rb
highlight_js.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
hijack.rb DEV: Add more debugging context to onebox generation 2020-10-22 12:50:22 +08:00
homepage_constraint.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
html_prettify.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
html_to_markdown.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
http_language_parser.rb FIX: Include resolved locale in anonymous cache key (#10289) 2020-07-22 18:00:07 +01:00
image_sizer.rb
import_export.rb FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
inline_oneboxer.rb FEATURE: Site setting for blocking onebox of URLs that redirect (#16881) 2022-05-23 13:52:06 +03:00
introduction_updater.rb FIX: replace default welcome topic post with new value from wizard 2020-04-01 15:42:45 -04:00
js_locale_helper.rb FIX: Overridden MessageFormat fallbacks (#15855) 2022-02-08 12:31:08 +11:00
json_error.rb
letter_avatar.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
markdown_linker.rb
mem_info.rb
message_bus_diags.rb PERF: avoid shelling to get hostname aggressively 2020-02-18 15:13:19 +11:00
method_profiler.rb DEV: Add output_sql_to_stderr! to MethodProfiler (#12445) 2021-03-19 17:48:30 +10:00
mini_sql_multisite_connection.rb DEV: Use MiniSql ActiveRecordPostgres adapter (#15767) 2022-02-03 10:00:28 +00:00
mobile_detection.rb FIX: include crawler content on old mobile browsers (#16387) 2022-04-06 11:09:12 +01:00
new_post_manager.rb SECURITY: Escape watched word in error message (#14434) 2021-09-24 11:55:15 +03:00
new_post_result.rb DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
notification_levels.rb
onebox.rb DEV: Don’t patch Sanitize::Config 2022-04-06 17:10:51 +02:00
oneboxer.rb FEATURE: Site setting for blocking onebox of URLs that redirect (#16881) 2022-05-23 13:52:06 +03:00
onpdiff.rb
pbkdf2.rb Use Xorcist.xor! instead of refinements since Ruby 3.2+ removes Refinment-include (#15694) 2022-02-03 16:19:30 +11:00
permalink_constraint.rb
pinned_check.rb
plain_text_to_markdown.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
plugin.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
plugin_gem.rb DEV: Don't load bundler when installing plugin gem. (#16117) 2022-03-07 13:20:43 +08:00
post_action_creator.rb DEV: Remove PostAction/UserAction bookmark refs (#16681) 2022-05-10 10:42:18 +10:00
post_action_destroyer.rb FIX: Background like count update didn't account for own user actions (#16688) 2022-05-09 17:23:39 -03:00
post_action_result.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
post_creator.rb FIX: Update user stat counts when post/topic visibility changes. (#15883) 2022-02-11 09:00:58 +08:00
post_destroyer.rb FIX: avoid validations when destroying posts (#16049) 2022-02-25 11:20:54 +11:00
post_jobs_enqueuer.rb FIX: Do not send emails to mailing_list_mode subscribers for PMs (#14159) 2021-08-26 15:16:35 +10:00
post_locker.rb
post_merger.rb FEATURE: TL4 & category moderators can merge posts (#12843) 2021-04-27 18:24:27 +02:00
post_revisor.rb FIX: Don't advance draft sequence when editing topic title (#16002) 2022-02-23 10:39:54 +03:00
presence_channel.rb DEV: Fix random typos (#16066) 2022-02-28 10:20:58 +08:00
pretty_text.rb DEV: Drop lodash (#16110) 2022-03-06 18:15:25 +01:00
promotion.rb FIX: check if BasicBadge is enabled for TL1 welcome message (#13983) 2021-08-11 08:39:25 +10:00
quote_comparer.rb FIX: handle quote rendering for external Discourse instance (#16722) 2022-05-12 10:07:43 -05:00
rake_helpers.rb Try fix upload_spec flakys and remove logging from tasks/uploads_spec 2020-02-18 15:08:58 +10:00
rate_limiter.rb DEV: Consolidate Redis evalsha logic into DiscourseRedis::EvalHelper (#15957) 2022-02-15 16:06:12 +00:00
read_only_mixin.rb DEV: New readonly mode. Only applies to non-staff (#16243) 2022-05-17 13:06:08 -05:00
redis_snapshot.rb FIX: add support for pipelined and multi redis commands (#16682) 2022-05-10 08:19:02 +10:00
require_dependency_backward_compatibility.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
retrieve_title.rb FEATURE: Site setting for blocking onebox of URLs that redirect (#16881) 2022-05-23 13:52:06 +03:00
route_format.rb
route_matcher.rb REFACTOR: Introduce RouteMatcher class 2020-10-19 10:40:55 +01:00
rtl.rb
s3_cors_rulesets.rb DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
s3_helper.rb FEATURE: Direct S3 multipart uploads for backups (#14736) 2021-11-11 08:25:31 +10:00
s3_inventory.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
score_calculator.rb
screening_model.rb
search.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
secure_session.rb
shrink_uploaded_image.rb DEV: Improve `script/downsize_uploads.rb` (#13508) 2021-06-24 00:09:40 +02:00
sidekiq_logster_reporter.rb DEV: Move SidekiqLogsterReporter out of initializer (#16687) 2022-05-09 13:01:31 -05:00
site_icon_manager.rb PERF: Defer setting of distributed cache in more spots. 2021-06-04 09:13:18 +08:00
site_setting_extension.rb FIX: Resetting selectable avatars was failing (#16302) 2022-03-28 14:15:28 -04:00
slug.rb FIX: Make category slugs lowercase (#11277) 2021-01-12 17:28:33 +02:00
socket_server.rb
spam_handler.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
staff_constraint.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
staff_message_format.rb
suggested_topics_builder.rb
svg_sprite.rb FIX: uses tippy for popover (#15409) 2022-05-02 17:10:26 +02:00
system_message.rb REFACTOR: Add `full_url` and `display_name` to `User` 2022-04-14 11:53:57 +02:00
temporary_db.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
temporary_redis.rb DEV: Introduce `TemporaryRedis` and unset `DISCOURSE_*` env vars in the `themes:isolated_test` rake task (#13401) 2021-06-23 07:38:43 +03:00
text_cleaner.rb FEATURE: Correctly convert topic title to uppercase and lowercase for Turkish default locale (#13115) 2021-05-24 18:13:30 +10:00
text_sentinel.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
theme_javascript_compiler.rb FEATURE: Allow theme tests to be run in production (take 2) (#12845) 2021-04-28 23:12:08 +03:00
theme_modifier_helper.rb Code review comments. 2021-06-21 11:06:58 +08:00
theme_settings_manager.rb FEATURE: Allow theme settings to request refresh (#15037) 2021-11-22 13:16:56 +01:00
theme_settings_parser.rb FEATURE: Allow theme settings to request refresh (#15037) 2021-11-22 13:16:56 +01:00
theme_translation_manager.rb
theme_translation_parser.rb
timeline_lookup.rb FIX: ensures timeline_lookup includes last tuple (#11829) 2021-01-25 11:30:59 +01:00
tiny_japanese_segmenter.rb FEATURE: Split up text segmentation for Chinese and Japanese. 2022-02-07 09:21:14 +08:00
topic_creator.rb FIX: Validate category tag restrictions before sending new topics to review (#16292) 2022-03-28 21:25:26 +03:00
topic_list_responder.rb DEV: Refactor draft attributes for `CategoryList` and `TopicList`. 2020-07-24 10:11:30 +08:00
topic_publisher.rb FIX: Use destroy_all instead of delete_all for shared drafts 2020-03-05 11:13:43 -08:00
topic_query.rb DEV: Remove PostAction/UserAction bookmark refs (#16681) 2022-05-10 10:42:18 +10:00
topic_query_params.rb FIX: Build correct topic list filter (#11473) 2020-12-11 14:20:48 +02:00
topic_retriever.rb FEATURE: Fallback to system users when creating new TopicEmbed (#12386) 2021-03-15 11:58:53 -03:00
topic_subtype.rb
topic_upload_security_manager.rb DEV: Add security_last_changed_at and security_last_changed_reason to uploads (#11860) 2021-01-29 09:03:44 +10:00
topic_view.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
topics_bulk_action.rb FEATURE Add reset bump dates bulk action (#16885) 2022-05-22 12:32:55 -04:00
trust_level.rb FIX: Don't store translated trust level names in anonymous cache (#13224) 2021-06-01 22:11:48 +02:00
turbo_tests.rb FIX: Make thumbnail tests start with a clean slate (#15216) 2021-12-07 13:07:45 -06:00
twitter_api.rb FIX: Replace Twitter handles one at a time (#15870) 2022-02-09 13:54:02 +02:00
unicorn_logstash_patch.rb DEV: Fix lint. 2020-07-21 15:55:03 +08:00
unread.rb FEATURE: Add last visit indication to topic view page. (#13471) 2021-07-05 14:17:31 +08:00
upload_creator.rb FIX: Blurry onebox favicon images (#15258) 2021-12-10 12:25:50 -07:00
upload_fixer.rb
upload_markdown.rb DEV: Expand UploadMarkdown generation capabilities (#15930) 2022-02-14 15:48:27 +10:00
upload_recovery.rb FIX: Support Ruby 3 keyword arguments 2021-10-05 11:25:00 -04:00
upload_security.rb PERF: Speed up secure media and ACL sync rake tasks (#16849) 2022-05-23 13:14:11 +10:00
url_helper.rb FEATURE: revert disallowing putting URLs in titles for TL0 users (#13970) 2021-08-06 20:07:42 +04:00
user_lookup.rb REVERT "FIX: do not show private group flair on user avatars" (#13991) 2021-08-10 17:25:11 +05:30
user_name_suggester.rb FEATURE: when suggesting usernames skip input that consist entirely of disallowed characters (#15368) 2021-12-21 21:13:05 +04:00
vary_header.rb FIX: Include the Vary:Accept header on all Accept-based responses (#14647) 2021-10-25 12:53:50 +01:00
version.rb Version bump to v2.9.0.beta4 (#16477) 2022-04-14 10:10:23 -04:00
webauthn.rb FEATURE: RS512, RS384 and RS256 COSE algorithms (#15804) 2022-02-08 14:07:47 +02:00
wizard.rb DEV: Allow plugins to add wizard steps after specific steps (#9315) 2020-04-01 08:36:50 -05:00