Commit Graph

272 Commits

Author SHA1 Message Date
Robin Ward c150566506
Migrate pretty-text to `.js` extensions (#9243) 2020-03-20 09:55:42 -04:00
Sam Saffron 145c90419d
DEV: increase timeout for initial context eval
Due to JS refactors our initial payload transpile is taking a bit longer
instead of failing the test suite attempt to give this a bit more time
2020-03-20 15:48:25 +11:00
Robin Ward a3f0543f99
Support for transpiling `.js` files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
Robin Ward 041168c9b6 FIX: Group mentions were not being cooked the same was as previewed
If a group mention could be notified on preview it was given an `<a>`
tag with the `.notify` class. When cooked it would display differently.
This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
Penar Musaraj 708190802c FIX: Strip video oneboxes from excerpts
Follows up on 49843f327e and removes onebox markup for videos too in the excerpt, this was previously being counted as part of the excerpt length.
2020-02-17 13:52:23 -05:00
Jarek Radosz 6cfd16656f
FIX: Ignore group mentions inside quotes (#8905)
Also includes:
* DEV: Reuse found elements
2020-02-10 18:31:42 +01:00
Penar Musaraj 49843f327e
FIX: Strip audio/video content from excerpt (#8881) 2020-02-06 15:08:13 -05:00
Martin Brennan 1150cd4621
FIX: Stop secure media URLs being censored too liberally in emails (#8817)
For example /t/ URLs were being replaced if they contained secure-media-uploads so if you made a topic called "Secure Media Uploads Are Cool" the View Topic link in the user notifications would be stripped out.

Refactored code so this secure URL detection happens in one place.
2020-01-30 16:19:14 +10:00
Martin Brennan 45b37a8bd1
FIX: Resolve pull hotlinked image and broken link issues for secure media URLs (#8777)
When pull_hotlinked_images tried to run on posts with secure media (which had already been downloaded from external sources) we were getting a 404 when trying to download the image because the secure endpoint doesn't allow anon downloads.

Also, we were getting into an infinite loop of pull_hotlinked_images because the job didn't consider the secure media URLs as "downloaded" already so it kept trying to download them over and over.

In this PR I have also refactored secure-media-upload URL checks and mutations into single source of truth in Upload, adding a SECURE_MEDIA_ROUTE constant to check URLs against too.
2020-01-24 11:59:30 +10:00
Gerhard Schlager 9ebb69e8eb FIX: Respect `enable_inline_emoji_translation` setting in titles 2019-12-03 20:39:11 +01:00
Penar Musaraj 102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Robin Ward 8d34f4bbd9 Revert "Revert Ember.run refactors"
This reverts commit fcb1ca52f9.
2019-10-30 09:48:24 -04:00
Robin Ward fcb1ca52f9 Revert Ember.run refactors
This reverts commit 5ca60fcb6b.
2019-10-29 17:10:47 -04:00
Robin Ward cfa5d123a3 Temporary fix: Don't use ember imports in pretty text 2019-10-29 16:45:05 -04:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Sam Saffron e9cf0de588 FEATURE: do not replace &harr; with an emoji
&harr; technically can be an emoji, but usually will not be used in that way
2019-08-30 15:06:23 +10:00
David Taylor 39e0442de9 FIX: Various watched words improvements
- Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback)
- Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins
- Censor tests are moved to ruby, to ensure everything works end-to-end
- If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-08-02 15:29:12 +01:00
Dan Ungureanu 9f5cfa192e
FEATURE: Allow Markdown in post notices. (#7864) 2019-07-09 14:42:02 +03:00
Guo Xiang Tan 1991af2abb DEV: Switch `InlineUploads` to a regexp based implementation. 2019-06-04 15:54:25 +08:00
Bianca Nenciu 227c45107d FEATURE: Implement Onebox for posts including polls. (#7539) 2019-05-29 17:05:52 +02:00
Guo Xiang Tan f0620e7118 FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Penar Musaraj 7c9fb95c15 Temporarily revert "FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD. (#7603)"
This reverts commit b1d3c678ca.

We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan b1d3c678ca FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD. (#7603) 2019-05-28 11:18:21 -04:00
Gerhard Schlager e224100023 FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled 2019-05-22 11:44:39 +02:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Arpit Jalan 9758857a78 FIX: escape Vimeo iframe URL 2019-05-09 23:01:50 +05:30
Arpit Jalan 66582ed956 FIX: get vimeo URL from `data-original-href` iframe attribute 2019-04-26 17:12:07 +05:30
Bianca Nenciu b5008586c5 DEV: Remove HTML parser from Tautologistics. (#7344) 2019-04-10 11:21:22 +02:00
Tim Lange f7b156ffbd UX: Better emoji escaping for topic title (#7218)
* FIX: Fixed failing discourse-prometheus-alert-receiver plugin specs
2019-03-21 09:11:33 +01:00
Régis Hanol 6fb49e74a1
Revert "UX: Better emoji escaping for topic title (#7176)" (#7201)
This reverts commit 0d9bc0aaa6.
2019-03-19 12:02:47 +01:00
Tim Lange 0d9bc0aaa6 UX: Better emoji escaping for topic title (#7176) 2019-03-19 09:33:10 +01:00
Joffrey JAFFEUX d32557ea32 Revert "FIX: Better emoji escaping for topic title"
This reverts commit 35426b5ad6.
2019-03-13 13:02:56 +01:00
Tim Lange 35426b5ad6 FIX: Better emoji escaping for topic title
This commit also puts emojiVersion in its own erb file.
2019-03-13 11:17:59 +01:00
Tim Lange 3ff0800e50 FIX: Fixed custom emoji circumventing "max emojis in topic title" set… (#7116) 2019-03-06 12:49:17 +01:00
Régis Hanol 664e90bd17 FIX: ensure local images use local CDN when uploads are stored on S3
When the S3 store was enabled, we were only applying the S3 CDN.
So all images stored locally, like the emojis, were never put on the local CDN.

Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls'
in order to get final URLs.

I also removed the unnecessary PrettyText.add_s3_cdn method since this is already
handled in the CookedPostProcessor.
2019-02-20 19:24:38 +01:00
Penar Musaraj 421d47da1e FIX: user and group mentions in subfolder installs 2019-01-29 21:54:29 -05:00
David Taylor afcf149c34 FIX: Fix mentions for mixed case group names 2018-11-26 15:34:56 +00:00
Guo Xiang Tan 57e2f4990d
PERF: Move processing of inline onebox out of V8 context. (#6658) 2018-11-26 09:21:38 +08:00
Guo Xiang Tan 7f7464e3e6 DEV: Remove unused variable. 2018-11-23 08:32:29 +08:00
Guo Xiang Tan 4752ddc908 Avoid DB query when there are no mentions. 2018-11-22 16:44:17 +08:00
Guo Xiang Tan 28a6cf8228 FIX: Mention lookup should be case insensitive. 2018-11-22 16:32:56 +08:00
Guo Xiang Tan 3f636b2d19 FIX: Check whether group is mentionable by user when cooking post. 2018-11-22 16:16:33 +08:00
Guo Xiang Tan 672e95bcb4 FIX: Staged users should not be mentionable. 2018-11-22 15:00:46 +08:00
Guo Xiang Tan c5a70eca6e
PERF: Move mention lookups out of the V8 context. (#6640)
We were looking up each mention one by one without any form of caching and that results
in a problem somewhat similar to an N+1. When we have to do alot of DB
lookups, it also increased the time spent in the V8 context which may
eventually lead to a timeout. The change here makes it such that mention lookups only does a single
DB query per post that happens outside of the V8 context.
2018-11-22 14:28:48 +08:00
Vinoth Kannan 24a14af15a FIX: Respect invalidate_oneboxes option for inline oneboxes 2018-09-03 22:33:43 +05:30
Régis Hanol de92913bf4 FIX: store the topic links using the cooked upload url 2018-08-14 12:23:32 +02:00
Neil Lalonde 4b2e42757d fix indent 2018-06-08 13:56:20 -04:00
Neil Lalonde b8cf0788c6 FIX: broken mailto href's in emails 2018-06-08 13:11:58 -04:00
Gerhard Schlager ea7ffac17e FIX: Paths used by PrettyText were not always initialized 2018-05-23 22:16:59 +02:00
Régis Hanol 6a006b3646 FIX: format posts for embedded comments as we do for emails 2018-05-09 19:24:44 +02:00
Régis Hanol 738f3f5229 Load missing libraries for server-side PrettyText 2018-05-05 11:21:07 +02:00
Joffrey JAFFEUX 45f657336e
FEATURE: adds support for loading existing core asset in pretty text 2018-04-10 08:37:16 +02:00
Arpit Jalan b75b6de982 FIX: respect nofollow settings for onebox links 2018-03-26 18:21:16 +05:30
Neil Lalonde 3313072957 Remove censored_pattern site setting, which is replaced by watched words 2018-02-26 16:29:27 -05:00
Sam ee0d3f15c1 FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify
New site settings:

enable_markdown_linkify: which is default on, auto links https:// and http:// and mail://

markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-02-01 13:22:38 +11:00
Robin Ward f8164956dd Add quote and mention support for username formatters 2017-11-20 16:28:03 -05:00
ckeboss 93633865d9 Adds primary user group as a class to quote (#5285)
* Adds primary user group as a class to quote

This feature addition will add the class `group-PRIMARY_USER_GROUP` to
the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group
of the user being quoted. This is similar to the class that is added to
a `topic-post`.

* Remove trailing whitespace

* Fix avatar in test

* Address PR comments

* Fix trailing whitespace
2017-11-03 09:51:40 -04:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan 23b787e0a6 Require dependency otherwise it causes Sidekiq to lock up in development. 2017-09-25 13:48:59 +08:00
Sam bcf7dc38c2 FEATURE: server side support for upload:// markdown
This allows uploads to be specified using short sha1 hash instead of full
URL

Client side change is pending
2017-08-22 11:46:23 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Neil Lalonde 24cb950432 FEATURE: Watched Words: when posts contain words, do one of flag, require approval, censor, or block 2017-07-26 11:01:09 -04:00
Sam e1ce47a901 Pass the full CommonMark spec 2017-07-21 13:20:52 -04:00
Robin Ward 3882722195 FEATURE: Inline (Mini) Oneboxing
see:
https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400?source_topic_id=66066
2017-07-20 15:38:04 -04:00
Guo Xiang Tan ac9c8ccf3b Fix broken specs. 2017-07-20 13:17:45 +09:00
Guo Xiang Tan 0edb0018ff Dispose of heap when we reset the context. 2017-07-20 13:10:56 +09:00
Sam Saffron d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam 79a084dd58 Revert "remove old markdown engine work-in-progress"
This reverts commit ee470b5317.
2017-07-12 18:10:51 -04:00
Sam bcbb9f208d Revert "Integrate new engine, correct old specs"
This reverts commit f1b38ba4fb.
2017-07-12 18:10:07 -04:00
Sam f1b38ba4fb Integrate new engine, correct old specs
corrects edge cases with

- full quotes
- [url] with nested tags
- engine overrides
- onebox applying to non http srcs
2017-07-12 17:44:40 -04:00
Sam Saffron ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
Sam 98e03b04b5 Don't depend on imports for md extensions 2017-07-11 16:48:25 -04:00
Sam cfbda863e8 implement unicode emoji replacements 2017-06-28 13:47:30 -04:00
Sam da5ccd2000 correctly hunt for console.log 2017-06-23 15:24:20 -04:00
Sam 234694b50f Feature: CommonMark support
This adds the markdown.it engine to Discourse.
https://github.com/markdown-it/markdown-it

As the migration is going to take a while the new engine is default
disabled. To enable it you must change the hidden site setting:
enable_experimental_markdown_it.

This commit is a squash of many other commits, it also includes some
improvements to autospec (ability to run plugins), and a dev dependency
on the og gem for html normalization.
2017-06-23 12:01:33 -04:00
Robin Ward 564eb8c20f Allow plugins to add vendored files for the text pipeline 2017-04-18 17:59:05 -04:00
Régis Hanol ba115480ba FIX: wasn't extracting links to quoted posts 2017-02-06 14:45:04 +01:00
Sam f932cb51f3 FIX: stop stripping local onebox links from tracker
When a onebox was made to a local topic it was not tracked using link
tracker
2017-02-01 14:21:01 -05:00
Arpit Jalan 2d0c99636a do not add rel noreferrer 2016-11-20 18:19:14 +05:30
Arpit Jalan 7cb76f7333 FIX: add rel noopener and noreferrer in addition to nofollow 2016-11-20 17:07:27 +05:30
Régis Hanol a0f1090d79 FIX: custom emojis leaking over multisites 2016-11-17 19:35:39 +01:00
Robin Ward 0f9e45f283 Allow the loader to understand when files are moved 2016-11-15 11:41:16 -05:00
Neil Lalonde 86522a52b7 FEATURE: add censored_pattern setting to censor posts using regex 2016-11-08 16:39:26 -05:00
Régis Hanol 78cd42943f FEATURE: add 'emoji-custom' class to custom emojis 2016-11-08 16:36:09 +01:00
Robin Ward 3c12dd6549 FIX: Consider lazyYT divs as links when extracting 2016-09-22 16:50:24 -04:00
Vinoth Kannan 648c8d27a6 chomp before regex manifest 2016-08-25 17:15:29 +05:30
Robin Ward 414388b94f FIX: Allow disabling of sanitization (for email customiaztions) 2016-08-11 14:59:20 -04:00
Robin Ward d763ce08c0 FIX: Travis failure 2016-08-10 13:20:39 -04:00
Guo Xiang Tan c3cab98998 FEATURE: Admins should be able to create polls even when plugin is disabled. 2016-07-13 18:05:19 +08:00
Robin Ward a546395397 REFACTOR: Migrate markdown functionality in ES6 2016-07-11 12:57:05 -04:00
Régis Hanol 5169bcdb6e FIX: httpshttps ultra secure URLs 2016-06-30 16:55:01 +02:00
Sam b15f6bd211 FIX: s3 cdn urls not remapped correctly 2016-06-30 18:58:38 +10:00
Régis Hanol f3905fd99a FIX: S3 CDN wasn't applied to lightboxed images 2016-06-27 22:08:49 +02:00
James Kiesel 7a6bc3f1d7 Apply notification styles to mailing list email manually (#4283)
* Apply notification styles to mailing list email manually

* Fix failing spec
2016-06-21 20:42:30 +05:30
Neil Lalonde f13470b96b Use db schema for tags instead of plugin store and custom fields 2016-05-26 14:29:48 -04:00
Arpit Jalan 36ba5f6716 FIX: broken onebox avatar image 2016-05-26 07:57:42 +05:30
Sam 695773db1c FEATURE: upgrade from therubyracer to mini_racer
This pushes our internal V8 JavaScript engine from Chrome 32 to 50.

It also resolves some long standing issues we had with the old wrapper.
2016-05-23 09:57:15 +10:00
Régis Hanol c012b18601 FIX: sending email wasn't working anymore when a plugin used 'reduce_cooked' (cc @gdpelican) 2016-05-21 20:13:00 +02:00
James Kiesel feffe23cc5 FEATURE: More granular mailing list mode (#4068)
* Rearrange frontend to account for mailing list mode

* Allow update of user preference for mailing list frequency

* Add mailing list frequency estimate

* Simplify frequency estimate; disable activity summary for mailing list mode

* Remove combined updates

* Add specs for enqueue mailing list mode job

* Write mailing list method for mailer

* Fix linting error

* Account for stale topics

* Add translations for default mailing list setting

* One query for mailing list topics

* Fix failing spec

* WIP

* Flesh out html template

* First pass at text-based mailing list summary

* Add user avatar

* Properly format posts for mailing list

* Move make_all_links_absolute into Email::Styles

* Apply first_seen_at to user

* Send mailing list email summary hourly based on first_seen_at

* Branch and test cleanup

* Use existing mailing list mode estimate

* Fix failing specs
2016-05-21 15:17:54 +02:00
Sam 3f9ed76dad no need to use discourse event here 2016-05-02 12:01:15 +10:00