Discource-C
/
discourse
mirror of https://github.com/discourse/discourse.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
discourse/spec/lib/pretty_text_spec.rb

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2728 lines
105 KiB
Ruby
Raw Normal View History

DEV: use #frozen_string_literal: true on all spec This change both speeds up specs (less strings to allocate) and helps catch cases where methods in Discourse are mutating inputs. Overall we will be migrating everything to use #frozen_string_literal: true it will take a while, but this is the first and safest move in this direction
2019-04-29 20:27:42 -04:00
# frozen_string_literal: true
Initial release of Discourse
2013-02-05 14:16:51 -05:00
require "pretty_text"
Add RSpec 4 compatibility (#17652) * Remove outdated option https://github.com/rspec/rspec-core/commit/04078317ba6577699d06cf4dccf014254dcde7a6 * Use the non-globally exposed RSpec syntax https://github.com/rspec/rspec-core/pull/2803 * Use the non-globally exposed RSpec syntax, cont https://github.com/rspec/rspec-core/pull/2803 * Comply to strict predicate matchers See: - https://github.com/rspec/rspec-expectations/pull/1195 - https://github.com/rspec/rspec-expectations/pull/1196 - https://github.com/rspec/rspec-expectations/pull/1277
2022-07-27 22:27:38 -04:00
RSpec.describe PrettyText do
PERF: Speed up tests (#15214)
2021-12-07 13:45:58 -05:00
fab!(:user) { Fabricate(:user) }
fab!(:post) { Fabricate(:post) }
Initial release of Discourse
2013-02-05 14:16:51 -05:00
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 08:27:28 -04:00
before { SiteSetting.enable_markdown_typographer = false }
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
def n(html)
PERF: remove oga gem oga gem is automatically required by the aws gem the oga gem retains about 1mb of memory, aws now uses nokogiri This also removes the html normalize from the pretty text specs that was a fair bit buggy as the polls test shows.
2018-02-14 22:35:20 -05:00
html.strip
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
end
def cook(*args)
PERF: remove oga gem oga gem is automatically required by the aws gem the oga gem retains about 1mb of memory, aws now uses nokogiri This also removes the html normalize from the pretty text specs that was a fair bit buggy as the polls test shows.
2018-02-14 22:35:20 -05:00
PrettyText.cook(*args)
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
end
FIX: remove meta data from lightbox in both excerpt (html & text)
2014-11-05 14:37:00 -05:00
let(:wrapped_image) do
"<div class=\"lightbox-wrapper\"><a href=\"//localhost:3000/uploads/default/4399/33691397e78b4d75.png\" class=\"lightbox\" title=\"Screen Shot 2014-04-14 at 9.47.10 PM.png\"><img src=\"//localhost:3000/uploads/default/_optimized/bd9/b20/bbbcd6a0c0_655x500.png\" width=\"655\" height=\"500\"><div class=\"meta\">\n<span class=\"filename\">Screen Shot 2014-04-14 at 9.47.10 PM.png</span><span class=\"informations\">966x737 1.47 MB</span><span class=\"expand\"></span>\n</div></a></div>"
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
end
FIX: remove meta data from lightbox in both excerpt (html & text)
2014-11-05 14:37:00 -05:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
describe "Quoting" do
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with avatar" do
FIX: `default_avatars` wasn't being used for some server side templates
2015-06-26 13:37:50 -04:00
let(:default_avatar) do
"//test.localhost/uploads/default/avatars/42d/57c/46ce7ee487/{size}.png"
custom avatar support
2013-08-13 16:08:29 -04:00
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
before { User.stubs(:default_template).returns(default_avatar) }
DEV: Update username in new quote format - Part 1 (#22032) When we introduced the new quote format with full-name display name: ``` [quote="Ted Johansson, post:1, topic:2, username:ted"] we overlooked the code responsible for rewriting quotes when a user's name is changed. ``` The functional part of this change adds support for the new quote format in the code that updates quotes when a user's username changes. See the test case in `spec/services/username_changer_spec.rb` for the details. In addition, this change adds a regression test for PrettyText to cover the new quote format, and extracts the code responsible for rewriting raw and cooked quotes into its own `QuoteRewriter` class. The functionality of the latter is tested through the tests in `spec/services/username_changer_spec.rb`.
2023-06-14 04:14:11 -04:00
it "correctly extracts usernames from the new quote format" do
topic = Fabricate(:topic, title: "this is a test topic :slight_smile:")
expected = <<~HTML
<aside class="quote no-group" data-username="codinghorror" data-post="2" data-topic="#{topic.id}">
<div class="title">
<div class="quote-controls"></div>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
<a href="http://test.localhost/t/this-is-a-test-topic/#{topic.id}/2">This is a test topic <img width="20" height="20" src="/images/emoji/twitter/slight_smile.png?v=#{Emoji::EMOJI_VERSION}" title="slight_smile" loading="lazy" alt="slight_smile" class="emoji"></a></div>
DEV: Update username in new quote format - Part 1 (#22032) When we introduced the new quote format with full-name display name: ``` [quote="Ted Johansson, post:1, topic:2, username:ted"] we overlooked the code responsible for rewriting quotes when a user's name is changed. ``` The functional part of this change adds support for the new quote format in the code that updates quotes when a user's username changes. See the test case in `spec/services/username_changer_spec.rb` for the details. In addition, this change adds a regression test for PrettyText to cover the new quote format, and extracts the code responsible for rewriting raw and cooked quotes into its own `QuoteRewriter` class. The functionality of the latter is tested through the tests in `spec/services/username_changer_spec.rb`.
2023-06-14 04:14:11 -04:00
<blockquote>
<p>ddd</p>
</blockquote>
</aside>
HTML
expect(
cook(
"[quote=\"Jeff, post:2, topic:#{topic.id}, username:codinghorror\"]\nddd\n[/quote]",
topic_id: 1,
),
).to eq(n(expected))
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "do off topic quoting with emoji unescape" do
topic = Fabricate(:topic, title: "this is a test topic :slight_smile:")
expected = <<~HTML
FIX: Quoting a quote preserves the original post information (#8746) Let's say post #2 quotes post number #1. If a user decides to quote the quote in post #2, it should keep the information of post #1 ("user_1, post: 1, topic: X"), instead of replacing with current post info ("user_2, post: 2, topic: X").
2020-01-22 09:10:23 -05:00
<aside class="quote no-group" data-username="EvilTrout" data-post="2" data-topic="#{topic.id}">
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<div class="title">
PERF: remove oga gem oga gem is automatically required by the aws gem the oga gem retains about 1mb of memory, aws now uses nokogiri This also removes the html normalize from the pretty text specs that was a fair bit buggy as the polls test shows.
2018-02-14 22:35:20 -05:00
<div class="quote-controls"></div>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
<a href="http://test.localhost/t/this-is-a-test-topic/#{topic.id}/2">This is a test topic <img width="20" height="20" src="/images/emoji/twitter/slight_smile.png?v=#{Emoji::EMOJI_VERSION}" title="slight_smile" loading="lazy" alt="slight_smile" class="emoji"></a></div>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<blockquote>
PERF: remove oga gem oga gem is automatically required by the aws gem the oga gem retains about 1mb of memory, aws now uses nokogiri This also removes the html normalize from the pretty text specs that was a fair bit buggy as the polls test shows.
2018-02-14 22:35:20 -05:00
<p>ddd</p>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
</blockquote>
</aside>
HTML
expect(
cook("[quote=\"EvilTrout, post:2, topic:#{topic.id}\"]\nddd\n[/quote]", topic_id: 1),
).to eq(n(expected))
custom avatar support
2013-08-13 16:08:29 -04:00
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with emojis" do
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
let(:md) { <<~MD }
> This is a quote with a regular emoji :upside_down_face:
> This is a quote with an emoji shortcut :)
> This is a quote with a Unicode emoji 😎
MD
it "does not unescape emojis when emojis are disabled" do
SiteSetting.enable_emoji = false
html = <<~HTML
<blockquote>
<p>This is a quote with a regular emoji :upside_down_face:</p>
</blockquote>
<blockquote>
<p>This is a quote with an emoji shortcut :)</p>
</blockquote>
<blockquote>
<p>This is a quote with a Unicode emoji 😎</p>
</blockquote>
HTML
expect(cook(md)).to eq(html.strip)
end
it "does not convert emoji shortcuts when emoji shortcuts are disabled" do
SiteSetting.enable_emoji_shortcuts = false
html = <<~HTML
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a regular emoji <img src="/images/emoji/twitter/upside_down_face.png?v=#{Emoji::EMOJI_VERSION}" title=":upside_down_face:" class="emoji" alt=":upside_down_face:" loading="lazy" width="20" height="20"></p>
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
</blockquote>
<blockquote>
<p>This is a quote with an emoji shortcut :)</p>
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a Unicode emoji <img src="/images/emoji/twitter/sunglasses.png?v=#{Emoji::EMOJI_VERSION}" title=":sunglasses:" class="emoji" alt=":sunglasses:" loading="lazy" width="20" height="20"></p>
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
</blockquote>
HTML
expect(cook(md)).to eq(html.strip)
end
it "unescapes all emojis" do
html = <<~HTML
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a regular emoji <img src="/images/emoji/twitter/upside_down_face.png?v=#{Emoji::EMOJI_VERSION}" title=":upside_down_face:" class="emoji" alt=":upside_down_face:" loading="lazy" width="20" height="20"></p>
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with an emoji shortcut <img src="/images/emoji/twitter/slight_smile.png?v=#{Emoji::EMOJI_VERSION}" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a Unicode emoji <img src="/images/emoji/twitter/sunglasses.png?v=#{Emoji::EMOJI_VERSION}" title=":sunglasses:" class="emoji" alt=":sunglasses:" loading="lazy" width="20" height="20"></p>
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
</blockquote>
HTML
expect(cook(md)).to eq(html.strip)
end
FEATURE: line with only 1 to 3 emojis will now display as large emojis
2020-01-16 03:54:26 -05:00
it "adds an only-emoji class when a line has only one emoji" do
md = <<~MD
DEV: replaces huge generated emoji list by a simpler regex (#11053) Note that this commit is also fixing various mistakes in emojis. Some of them have been fixed manually in db.json/data.js/groups.json and will need to be fixed in emoji-db gem.
2021-04-22 02:43:06 -04:00
☹️
FEATURE: line with only 1 to 3 emojis will now display as large emojis
2020-01-16 03:54:26 -05:00
foo 😀
foo 😀 bar
:smile_cat:
:smile_cat: :smile_cat:
:smile_cat: :smile_cat: :smile_cat: :smile_cat:
baz? :smile_cat:
😀
😉 foo
😉 😉
😉 😉
😉 😉 😉
😉😉😉
😉 😉 😉
😉d😉 😉
😉 😉 😉d
😉😉😉😉
MD
html = <<~HTML
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p><img src="/images/emoji/twitter/frowning.png?v=#{Emoji::EMOJI_VERSION}" title=":frowning:" class="emoji only-emoji" alt=":frowning:" loading="lazy" width="20" height="20"><br>
foo <img src="/images/emoji/twitter/grinning.png?v=#{Emoji::EMOJI_VERSION}" title=":grinning:" class="emoji" alt=":grinning:" loading="lazy" width="20" height="20"><br>
foo <img src="/images/emoji/twitter/grinning.png?v=#{Emoji::EMOJI_VERSION}" title=":grinning:" class="emoji" alt=":grinning:" loading="lazy" width="20" height="20"> bar<br>
<img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji only-emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji only-emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji only-emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"><br>
baz? <img src="/images/emoji/twitter/smile_cat.png?v=#{Emoji::EMOJI_VERSION}" title=":smile_cat:" class="emoji" alt=":smile_cat:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/grinning.png?v=#{Emoji::EMOJI_VERSION}" title=":grinning:" class="emoji only-emoji" alt=":grinning:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> foo<br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji only-emoji" alt=":wink:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20">d​:wink: <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"><br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"> <img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20">d<br>
<img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"><img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"><img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"><img src="/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
FEATURE: line with only 1 to 3 emojis will now display as large emojis
2020-01-16 03:54:26 -05:00
HTML
expect(cook(md)).to eq(html.strip)
end
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
it "does use emoji CDN when enabled" do
SiteSetting.external_emoji_url = "https://emoji.cdn.com"
html = <<~HTML
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a regular emoji <img src="https://emoji.cdn.com/twitter/upside_down_face.png?v=#{Emoji::EMOJI_VERSION}" title=":upside_down_face:" class="emoji" alt=":upside_down_face:" loading="lazy" width="20" height="20"></p>
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with an emoji shortcut <img src="https://emoji.cdn.com/twitter/slight_smile.png?v=#{Emoji::EMOJI_VERSION}" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a Unicode emoji <img src="https://emoji.cdn.com/twitter/sunglasses.png?v=#{Emoji::EMOJI_VERSION}" title=":sunglasses:" class="emoji" alt=":sunglasses:" loading="lazy" width="20" height="20"></p>
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
</blockquote>
HTML
expect(cook(md)).to eq(html.strip)
end
it "does use emoji CDN when others CDNs are also enabled" do
set_cdn_url("https://cdn.com")
setup_s3
SiteSetting.s3_cdn_url = "https://s3.cdn.com"
SiteSetting.external_emoji_url = "https://emoji.cdn.com"
html = <<~HTML
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a regular emoji <img src="https://emoji.cdn.com/twitter/upside_down_face.png?v=#{Emoji::EMOJI_VERSION}" title=":upside_down_face:" class="emoji" alt=":upside_down_face:" loading="lazy" width="20" height="20"></p>
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with an emoji shortcut <img src="https://emoji.cdn.com/twitter/slight_smile.png?v=#{Emoji::EMOJI_VERSION}" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
</blockquote>
<blockquote>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p>This is a quote with a Unicode emoji <img src="https://emoji.cdn.com/twitter/sunglasses.png?v=#{Emoji::EMOJI_VERSION}" title=":sunglasses:" class="emoji" alt=":sunglasses:" loading="lazy" width="20" height="20"></p>
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180)
2021-03-02 14:04:16 -05:00
</blockquote>
HTML
expect(cook(md)).to eq(html.strip)
end
FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled
2019-05-21 10:56:51 -04:00
end
FIX: show generic title when quoting off-topic secure category posts
2018-12-02 13:22:40 -05:00
it "do off topic quoting of posts from secure categories" do
category = Fabricate(:category, read_restricted: true)
topic = Fabricate(:topic, title: "this is topic with secret category", category: category)
expected = <<~HTML
FIX: Quoting a quote preserves the original post information (#8746) Let's say post #2 quotes post number #1. If a user decides to quote the quote in post #2, it should keep the information of post #1 ("user_1, post: 1, topic: X"), instead of replacing with current post info ("user_2, post: 2, topic: X").
2020-01-22 09:10:23 -05:00
<aside class="quote no-group" data-username="maja" data-post="3" data-topic="#{topic.id}">
FIX: show generic title when quoting off-topic secure category posts
2018-12-02 13:22:40 -05:00
<div class="title">
<div class="quote-controls"></div>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
<a href="http://test.localhost/t/#{topic.id}/3">#{I18n.t("on_another_topic")}</a></div>
FIX: show generic title when quoting off-topic secure category posts
2018-12-02 13:22:40 -05:00
<blockquote>
<p>I have nothing to say.</p>
</blockquote>
</aside>
HTML
expect(
cook(
"[quote=\"maja, post:3, topic:#{topic.id}\"]\nI have nothing to say.\n[/quote]",
topic_id: 1,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
FIX: show generic title when quoting off-topic secure category posts
2018-12-02 13:22:40 -05:00
).to eq(n(expected))
end
DEV: Add force_quote_link option to PrettyText (#16034) This option will make it so the [quote] bbcode will always include the HTML link to the quoted post, even if a topic_id is not provided in the PrettyText#cook options. This is so [quote] bbcode can be used in other places, like chat messages, that always need the link and do not have an "off-topic" ID to use.
2022-02-23 01:13:46 -05:00
it "do off topic quoting with the force_quote_link opt and no topic_id opt provided" do
topic = Fabricate(:topic, title: "This is an off-topic topic")
expected = <<~HTML
<aside class="quote no-group" data-username="maja" data-post="3" data-topic="#{topic.id}">
<div class="title">
<div class="quote-controls"></div>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
<a href="http://test.localhost/t/this-is-an-off-topic-topic/#{topic.id}/3">#{topic.title}</a></div>
DEV: Add force_quote_link option to PrettyText (#16034) This option will make it so the [quote] bbcode will always include the HTML link to the quoted post, even if a topic_id is not provided in the PrettyText#cook options. This is so [quote] bbcode can be used in other places, like chat messages, that always need the link and do not have an "off-topic" ID to use.
2022-02-23 01:13:46 -05:00
<blockquote>
<p>I have nothing to say.</p>
</blockquote>
</aside>
HTML
cooked =
cook(
"[quote=\"maja, post:3, topic:#{topic.id}\"]\nI have nothing to say.\n[/quote]",
force_quote_link: true,
)
expect(cooked).to eq(n(expected))
end
FIX: allow bbcode to be unquoted for default
2017-07-24 18:36:17 -04:00
it "indifferent about missing quotations" do
md = <<~MD
[quote=#{user.username}, post:123, topic:456, full:true]
ddd
[/quote]
MD
html = <<~HTML
FIX: Quoting a quote preserves the original post information (#8746) Let's say post #2 quotes post number #1. If a user decides to quote the quote in post #2, it should keep the information of post #1 ("user_1, post: 1, topic: X"), instead of replacing with current post info ("user_2, post: 2, topic: X").
2020-01-22 09:10:23 -05:00
<aside class="quote no-group" data-username="#{user.username}" data-post="123" data-topic="456" data-full="true">
FIX: allow bbcode to be unquoted for default
2017-07-24 18:36:17 -04:00
<div class="title">
<div class="quote-controls"></div>
FEATURE: reduce avatar sizes to 6 from 20 (#21319) * FEATURE: reduce avatar sizes to 6 from 20 This PR introduces 3 changes: 1. SiteSetting.avatar_sizes, now does what is says on the tin. previously it would introduce a large number of extra sizes, to allow for various DPIs. Instead we now trust the admin with the size list. 2. When `avatar_sizes` changes, we ensure consistency and remove resized avatars that are not longer allowed per site setting. This happens on the 12 hourly job and limited out of the box to 20k cleanups per cycle, given this may reach out to AWS 20k times to remove things. 3.Our default avatar sizes are now "24|48|72|96|144|288" these sizes were very specifically picked to limit amount of bluriness introduced by webkit. Our avatars are already blurry due to 1px border, so this corrects old blur. This change heavily reduces storage required by forums which simplifies site moves and more. Co-authored-by: David Taylor <david@taylorhq.com>
2023-05-31 20:00:01 -04:00
<img loading="lazy" alt="" width="24" height="24" src="//test.localhost/uploads/default/avatars/42d/57c/46ce7ee487/48.png" class="avatar"> #{user.username}:</div>
FIX: allow bbcode to be unquoted for default
2017-07-24 18:36:17 -04:00
<blockquote>
<p>ddd</p>
</blockquote>
</aside>
HTML
expect(PrettyText.cook(md)).to eq(html.strip)
end
it "indifferent about curlies and no curlies" do
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
md = <<~MD
FIX: smart quote edge case with quotes
2017-07-24 12:21:49 -04:00
[quote=“#{user.username}, post:123, topic:456, full:true”]
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
ddd
[/quote]
MD
html = <<~HTML
FIX: Quoting a quote preserves the original post information (#8746) Let's say post #2 quotes post number #1. If a user decides to quote the quote in post #2, it should keep the information of post #1 ("user_1, post: 1, topic: X"), instead of replacing with current post info ("user_2, post: 2, topic: X").
2020-01-22 09:10:23 -05:00
<aside class="quote no-group" data-username="#{user.username}" data-post="123" data-topic="456" data-full="true">
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<div class="title">
<div class="quote-controls"></div>
FEATURE: reduce avatar sizes to 6 from 20 (#21319) * FEATURE: reduce avatar sizes to 6 from 20 This PR introduces 3 changes: 1. SiteSetting.avatar_sizes, now does what is says on the tin. previously it would introduce a large number of extra sizes, to allow for various DPIs. Instead we now trust the admin with the size list. 2. When `avatar_sizes` changes, we ensure consistency and remove resized avatars that are not longer allowed per site setting. This happens on the 12 hourly job and limited out of the box to 20k cleanups per cycle, given this may reach out to AWS 20k times to remove things. 3.Our default avatar sizes are now "24|48|72|96|144|288" these sizes were very specifically picked to limit amount of bluriness introduced by webkit. Our avatars are already blurry due to 1px border, so this corrects old blur. This change heavily reduces storage required by forums which simplifies site moves and more. Co-authored-by: David Taylor <david@taylorhq.com>
2023-05-31 20:00:01 -04:00
<img loading="lazy" alt="" width="24" height="24" src="//test.localhost/uploads/default/avatars/42d/57c/46ce7ee487/48.png" class="avatar"> #{user.username}:</div>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<blockquote>
<p>ddd</p>
</blockquote>
</aside>
HTML
expect(PrettyText.cook(md)).to eq(html.strip)
custom avatar support
2013-08-13 16:08:29 -04:00
end
it "trims spaces on quote params" do
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
md = <<~MD
[quote="#{user.username}, post:555, topic: 666"]
ddd
[/quote]
MD
html = <<~HTML
FIX: Quoting a quote preserves the original post information (#8746) Let's say post #2 quotes post number #1. If a user decides to quote the quote in post #2, it should keep the information of post #1 ("user_1, post: 1, topic: X"), instead of replacing with current post info ("user_2, post: 2, topic: X").
2020-01-22 09:10:23 -05:00
<aside class="quote no-group" data-username="#{user.username}" data-post="555" data-topic="666">
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<div class="title">
<div class="quote-controls"></div>
FEATURE: reduce avatar sizes to 6 from 20 (#21319) * FEATURE: reduce avatar sizes to 6 from 20 This PR introduces 3 changes: 1. SiteSetting.avatar_sizes, now does what is says on the tin. previously it would introduce a large number of extra sizes, to allow for various DPIs. Instead we now trust the admin with the size list. 2. When `avatar_sizes` changes, we ensure consistency and remove resized avatars that are not longer allowed per site setting. This happens on the 12 hourly job and limited out of the box to 20k cleanups per cycle, given this may reach out to AWS 20k times to remove things. 3.Our default avatar sizes are now "24|48|72|96|144|288" these sizes were very specifically picked to limit amount of bluriness introduced by webkit. Our avatars are already blurry due to 1px border, so this corrects old blur. This change heavily reduces storage required by forums which simplifies site moves and more. Co-authored-by: David Taylor <david@taylorhq.com>
2023-05-31 20:00:01 -04:00
<img loading="lazy" alt="" width="24" height="24" src="//test.localhost/uploads/default/avatars/42d/57c/46ce7ee487/48.png" class="avatar"> #{user.username}:</div>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<blockquote>
<p>ddd</p>
</blockquote>
</aside>
HTML
expect(PrettyText.cook(md)).to eq(html.strip)
custom avatar support
2013-08-13 16:08:29 -04:00
end
FIX: letter avatars in quotes on subfolders installs are broken
2017-06-23 17:12:06 -04:00
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with primary user group" do
Adds primary user group as a class to quote (#5285) * Adds primary user group as a class to quote This feature addition will add the class `group-PRIMARY_USER_GROUP` to the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group of the user being quoted. This is similar to the class that is added to a `topic-post`. * Remove trailing whitespace * Fix avatar in test * Address PR comments * Fix trailing whitespace
2017-11-03 09:51:40 -04:00
let(:default_avatar) do
"//test.localhost/uploads/default/avatars/42d/57c/46ce7ee487/{size}.png"
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
end
DEV: Prefabrication (test optimization) (#7414) * Introduced fab!, a helper that creates database state for a group It's almost identical to let_it_be, except: 1. It creates a new object for each test by default, 2. You can disable it using PREFABRICATION=0
2019-05-06 23:12:20 -04:00
fab!(:group) { Fabricate(:group) }
fab!(:user) { Fabricate(:user, primary_group: group) }
Adds primary user group as a class to quote (#5285) * Adds primary user group as a class to quote This feature addition will add the class `group-PRIMARY_USER_GROUP` to the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group of the user being quoted. This is similar to the class that is added to a `topic-post`. * Remove trailing whitespace * Fix avatar in test * Address PR comments * Fix trailing whitespace
2017-11-03 09:51:40 -04:00
before { User.stubs(:default_template).returns(default_avatar) }
it "adds primary group class to referenced users quote" do
topic = Fabricate(:topic, title: "this is a test topic")
expected = <<~HTML
FIX: Quoting a quote preserves the original post information (#8746) Let's say post #2 quotes post number #1. If a user decides to quote the quote in post #2, it should keep the information of post #1 ("user_1, post: 1, topic: X"), instead of replacing with current post info ("user_2, post: 2, topic: X").
2020-01-22 09:10:23 -05:00
<aside class="quote group-#{group.name}" data-username="#{user.username}" data-post="2" data-topic="#{topic.id}">
Adds primary user group as a class to quote (#5285) * Adds primary user group as a class to quote This feature addition will add the class `group-PRIMARY_USER_GROUP` to the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group of the user being quoted. This is similar to the class that is added to a `topic-post`. * Remove trailing whitespace * Fix avatar in test * Address PR comments * Fix trailing whitespace
2017-11-03 09:51:40 -04:00
<div class="title">
PERF: remove oga gem oga gem is automatically required by the aws gem the oga gem retains about 1mb of memory, aws now uses nokogiri This also removes the html normalize from the pretty text specs that was a fair bit buggy as the polls test shows.
2018-02-14 22:35:20 -05:00
<div class="quote-controls"></div>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
<img loading="lazy" alt="" width="24" height="24" src="//test.localhost/uploads/default/avatars/42d/57c/46ce7ee487/48.png" class="avatar"><a href="http://test.localhost/t/this-is-a-test-topic/#{topic.id}/2">This is a test topic</a></div>
Adds primary user group as a class to quote (#5285) * Adds primary user group as a class to quote This feature addition will add the class `group-PRIMARY_USER_GROUP` to the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group of the user being quoted. This is similar to the class that is added to a `topic-post`. * Remove trailing whitespace * Fix avatar in test * Address PR comments * Fix trailing whitespace
2017-11-03 09:51:40 -04:00
<blockquote>
PERF: remove oga gem oga gem is automatically required by the aws gem the oga gem retains about 1mb of memory, aws now uses nokogiri This also removes the html normalize from the pretty text specs that was a fair bit buggy as the polls test shows.
2018-02-14 22:35:20 -05:00
<p>ddd</p>
Adds primary user group as a class to quote (#5285) * Adds primary user group as a class to quote This feature addition will add the class `group-PRIMARY_USER_GROUP` to the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group of the user being quoted. This is similar to the class that is added to a `topic-post`. * Remove trailing whitespace * Fix avatar in test * Address PR comments * Fix trailing whitespace
2017-11-03 09:51:40 -04:00
</blockquote>
</aside>
HTML
expect(
cook(
"[quote=\"#{user.username}, post:2, topic:#{topic.id}\"]\nddd\n[/quote]",
topic_id: 1,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
Adds primary user group as a class to quote (#5285) * Adds primary user group as a class to quote This feature addition will add the class `group-PRIMARY_USER_GROUP` to the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group of the user being quoted. This is similar to the class that is added to a `topic-post`. * Remove trailing whitespace * Fix avatar in test * Address PR comments * Fix trailing whitespace
2017-11-03 09:51:40 -04:00
).to eq(n(expected))
end
end
FEATURE: support inline bbcode blocks eg: [quote]test[/quote]
2017-07-27 17:55:04 -04:00
it "can handle inline block bbcode" do
cooked = PrettyText.cook("[quote]te **s** t[/quote]")
html = <<~HTML
FIX: Broken specs too. Quotes are fragile!
2018-04-02 14:23:10 -04:00
<aside class="quote no-group">
FEATURE: support inline bbcode blocks eg: [quote]test[/quote]
2017-07-27 17:55:04 -04:00
<blockquote>
<p>te <strong>s</strong> t</p>
</blockquote>
</aside>
HTML
expect(cooked).to eq(html.strip)
end
FIX: [constructor] bbcode would cause markdown crash
2017-12-27 00:11:30 -05:00
it "handles bbcode edge cases" do
expect(PrettyText.cook "[constructor]\ntest").to eq("<p>[constructor]<br>\ntest</p>")
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle quote edge cases" do
FEATURE: support inline bbcode blocks eg: [quote]test[/quote]
2017-07-27 17:55:04 -04:00
expect(PrettyText.cook("[quote]abc\ntest\n[/quote]")).not_to include("aside")
expect(PrettyText.cook("[quote] \ntest\n[/quote] ")).to include("aside")
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(PrettyText.cook("a\n[quote]\ntest\n[/quote]\n\n\na")).to include("aside")
expect(PrettyText.cook("- a\n[quote]\ntest\n[/quote]\n\n\na")).to include("aside")
expect(PrettyText.cook("[quote]\ntest")).not_to include("aside")
expect(PrettyText.cook("[quote]\ntest\n[/quote]z")).not_to include("aside")
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
nested = <<~MD
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
[quote]
a
[quote]
b
[/quote]
c
[/quote]
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
MD
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
cooked = PrettyText.cook(nested)
expect(cooked.scan("aside").length).to eq(4)
expect(cooked.scan("quote]").length).to eq(0)
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with letter avatar" do
context "with subfolder" do
FIX: letter avatars in quotes on subfolders installs are broken
2017-06-23 17:12:06 -04:00
it "should have correct avatar url" do
DEV: improve usability of subfolder specs Previously people were not consistent about mocking which left internals in a fragile state when running subfolder specs. This introduces a simple helper `set_subfolder` which you can use to set the subfolder for the spec. It takes care of proper configuration of subfolder and teardown. ``` # usage set_subfolder "/my_amazing_subfolder" ``` You should no longer stub base_uri or global_settings
2019-11-15 00:48:24 -05:00
set_subfolder "/forum"
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
md = <<~MD
[quote="#{user.username}, post:123, topic:456, full:true"]
ddd
[/quote]
MD
expect(PrettyText.cook(md)).to include("/forum/letter_avatar_proxy")
FIX: letter avatars in quotes on subfolders installs are broken
2017-06-23 17:12:06 -04:00
end
end
custom avatar support
2013-08-13 16:08:29 -04:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
describe "Mentions" do
FIX: mentions broken after adding an <abbr> tag A previous shortcut used was not allowing for <abbr and other tags starting with a If <abbr> appeared anywhere in the text all mentions would fail to link
2018-07-04 19:27:11 -04:00
it "can handle mentions after abbr" do
expect(PrettyText.cook("test <abbr>test</abbr>\n\n@bob")).to eq(
"<p>test <abbr>test</abbr></p>\n<p><span class=\"mention\">@bob</span></p>",
)
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should handle 3 mentions in a row" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.cook("@hello @hello @hello"),
).to match_html "<p><span class=\"mention\">@hello</span> <span class=\"mention\">@hello</span> <span class=\"mention\">@hello</span></p>"
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
Correct mention edge cases @sam. was not correctly detected as a mention
2017-07-20 11:52:29 -04:00
it "can handle mention edge cases" do
expect(PrettyText.cook("hi\n@s")).to eq("<p>hi<br>\n<span class=\"mention\">@s</span></p>")
expect(PrettyText.cook("hi\n@ss")).to eq("<p>hi<br>\n<span class=\"mention\">@ss</span></p>")
expect(PrettyText.cook("hi\n@s.")).to eq("<p>hi<br>\n<span class=\"mention\">@s</span>.</p>")
expect(PrettyText.cook("hi\n@s.s")).to eq(
"<p>hi<br>\n<span class=\"mention\">@s.s</span></p>",
)
expect(PrettyText.cook("hi\n@.s.s")).to eq("<p>hi<br>\n@.s.s</p>")
end
PERF: Move mention lookups out of the V8 context. (#6640) We were looking up each mention one by one without any form of caching and that results in a problem somewhat similar to an N+1. When we have to do alot of DB lookups, it also increased the time spent in the V8 context which may eventually lead to a timeout. The change here makes it such that mention lookups only does a single DB query per post that happens outside of the V8 context.
2018-11-22 01:28:48 -05:00
it "handles user and group mentions correctly" do
%w[User user2].each { |username| Fabricate(:user, username: username) }
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
FIX: Group mentions were not being cooked the same was as previewed If a group mention could be notified on preview it was given an `<a>` tag with the `.notify` class. When cooked it would display differently. This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
Fabricate(:group, name: "Group", mentionable_level: Group::ALIAS_LEVELS[:everyone])
Fabricate(
:group,
name: "Group2",
mentionable_level: Group::ALIAS_LEVELS[:members_mods_and_admins],
)
PERF: Try to match users before groups. User mentions are more common than group mentions so this will allow us to avoid an extra query.
2018-11-16 03:41:20 -05:00
PERF: Move mention lookups out of the V8 context. (#6640) We were looking up each mention one by one without any form of caching and that results in a problem somewhat similar to an N+1. When we have to do alot of DB lookups, it also increased the time spent in the V8 context which may eventually lead to a timeout. The change here makes it such that mention lookups only does a single DB query per post that happens outside of the V8 context.
2018-11-22 01:28:48 -05:00
[
[
FIX: Mention lookup should be case insensitive.
2018-11-22 03:32:56 -05:00
"hi @uSer! @user2 hi",
'<p>hi <a class="mention" href="/u/user">@uSer</a>! <a class="mention" href="/u/user2">@user2</a> hi</p>',
PERF: Move mention lookups out of the V8 context. (#6640) We were looking up each mention one by one without any form of caching and that results in a problem somewhat similar to an N+1. When we have to do alot of DB lookups, it also increased the time spent in the V8 context which may eventually lead to a timeout. The change here makes it such that mention lookups only does a single DB query per post that happens outside of the V8 context.
2018-11-22 01:28:48 -05:00
],
[
FIX: Fix mentions for mixed case group names
2018-11-26 10:34:56 -05:00
"hi\n@user. @GROUP @somemention @group2",
FIX: Group mentions were not being cooked the same was as previewed If a group mention could be notified on preview it was given an `<a>` tag with the `.notify` class. When cooked it would display differently. This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
%Q|<p>hi<br>\n<a class="mention" href="/u/user">@user</a>. <a class="mention-group notify" href="/groups/group">@GROUP</a> <span class="mention">@somemention</span> <a class="mention-group" href="/groups/group2">@group2</a></p>|,
PERF: Move mention lookups out of the V8 context. (#6640) We were looking up each mention one by one without any form of caching and that results in a problem somewhat similar to an N+1. When we have to do alot of DB lookups, it also increased the time spent in the V8 context which may eventually lead to a timeout. The change here makes it such that mention lookups only does a single DB query per post that happens outside of the V8 context.
2018-11-22 01:28:48 -05:00
],
].each { |input, expected| expect(PrettyText.cook(input)).to eq(expected) }
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with subfolder" do
FIX: user and group mentions in subfolder installs
2019-01-29 21:54:29 -05:00
it "handles user and group mentions correctly" do
DEV: improve usability of subfolder specs Previously people were not consistent about mocking which left internals in a fragile state when running subfolder specs. This introduces a simple helper `set_subfolder` which you can use to set the subfolder for the spec. It takes care of proper configuration of subfolder and teardown. ``` # usage set_subfolder "/my_amazing_subfolder" ``` You should no longer stub base_uri or global_settings
2019-11-15 00:48:24 -05:00
set_subfolder "/forum"
FIX: user and group mentions in subfolder installs
2019-01-29 21:54:29 -05:00
Fabricate(:user, username: "user1")
Fabricate(:group, name: "groupA", mentionable_level: Group::ALIAS_LEVELS[:everyone])
input = "hi there @user1 and @groupA"
FIX: Group mentions were not being cooked the same was as previewed If a group mention could be notified on preview it was given an `<a>` tag with the `.notify` class. When cooked it would display differently. This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
expected =
'<p>hi there <a class="mention" href="/forum/u/user1">@user1</a> and <a class="mention-group notify" href="/forum/groups/groupa">@groupA</a></p>'
FIX: user and group mentions in subfolder installs
2019-01-29 21:54:29 -05:00
expect(PrettyText.cook(input)).to eq(expected)
end
end
FIX: Group mentions were not being cooked the same was as previewed If a group mention could be notified on preview it was given an `<a>` tag with the `.notify` class. When cooked it would display differently. This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
it "does not assign the notify class to a group that can't be mentioned" do
UX: Include public groups in mentionable groups set (#8516)
2019-12-12 06:13:40 -05:00
group =
Fabricate(
:group,
visibility_level: Group.visibility_levels[:members],
mentionable_level: Group::ALIAS_LEVELS[:nobody],
)
FIX: Check whether group is mentionable by user when cooking post.
2018-11-22 03:01:03 -05:00
expect(PrettyText.cook("test @#{group.name} test")).to eq(
FIX: Group mentions were not being cooked the same was as previewed If a group mention could be notified on preview it was given an `<a>` tag with the `.notify` class. When cooked it would display differently. This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
%Q|<p>test <a class="mention-group" href="/groups/#{group.name}">@#{group.name}</a> test</p>|,
)
end
it "assigns the notify class if the user can mention" do
group =
Fabricate(
:group,
visibility_level: Group.visibility_levels[:members],
mentionable_level: Group::ALIAS_LEVELS[:members_mods_and_admins],
)
expect(PrettyText.cook("test @#{group.name} test", user_id: Fabricate(:admin).id)).to eq(
%Q|<p>test <a class="mention-group notify" href="/groups/#{group.name}">@#{group.name}</a> test</p>|,
FIX: Check whether group is mentionable by user when cooking post.
2018-11-22 03:01:03 -05:00
)
end
FIX: Staged users should not be mentionable.
2018-11-22 02:00:46 -05:00
it "does not mention staged users" do
user = Fabricate(:user, staged: true)
expect(PrettyText.cook("something @#{user.username} something")).to eq(
%Q|<p>something <span class="mention">@#{user.username}</span> something</p>|,
)
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "when mentions are disabled" do
PERF: Move mention lookups out of the V8 context. (#6640) We were looking up each mention one by one without any form of caching and that results in a problem somewhat similar to an N+1. When we have to do alot of DB lookups, it also increased the time spent in the V8 context which may eventually lead to a timeout. The change here makes it such that mention lookups only does a single DB query per post that happens outside of the V8 context.
2018-11-22 01:28:48 -05:00
before { SiteSetting.enable_mentions = false }
it "should not convert mentions to links" do
expect(PrettyText.cook("hi @user")).to eq("<p>hi @user</p>")
end
PERF: Try to match users before groups. User mentions are more common than group mentions so this will allow us to avoid an extra query.
2018-11-16 03:41:20 -05:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle mentions inside a hyperlink" do
expect(PrettyText.cook("<a> @inner</a> ")).to match_html "<p><a> @inner</a></p>"
fix: support for hyphens in group name The group mention @ORG-team triggers notifications for the group @ORG. This fix changes the RegExp, so that the group name is correctly extracted. see: https://meta.discourse.org/t/group-mentions-that-begin-with-the-same-characters-may-be-incorrect/39892/12?u=rriemann
2016-02-23 14:57:54 -05:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle mentions inside a hyperlink" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(
PrettyText.cook("[link @inner](http://site.com)"),
).to match_html '<p><a href="http://site.com" rel="noopener nofollow ugc">link @inner</a></p>'
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
it "can handle a list of mentions" do
expect(PrettyText.cook("@a,@b")).to match_html(
'<p><span class="mention">@a</span>,<span class="mention">@b</span></p>',
)
end
it "should handle group mentions with a hyphen and without" do
expect(
PrettyText.cook("@hello @hello-hello"),
).to match_html "<p><span class=\"mention\">@hello</span> <span class=\"mention\">@hello-hello</span></p>"
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
it "should allow for @mentions to have punctuation" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.cook("hello @bob's @bob,@bob; @bob\"")).to match_html(
"<p>hello <span class=\"mention\">@bob</span>'s <span class=\"mention\">@bob</span>,<span class=\"mention\">@bob</span>; <span class=\"mention\">@bob</span>\"</p>",
)
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
FIX: medium URL with @ incorrectly handled as mention Also: - remove unused code - rename bbcode_ruler to bbcode.ruler - add md.core.textPostProcess.ruler to apply at end of chain (excluding links)
2017-07-17 16:21:47 -04:00
it "should not treat a medium link as a mention" do
expect(PrettyText.cook(". http://test/@sam")).not_to include("mention")
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
FEATURE: Add support for Unicode usernames and group names Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2019-04-23 06:22:47 -04:00
context "with Unicode usernames disabled" do
before { SiteSetting.unicode_usernames = false }
it "does not detect mention" do
expect(PrettyText.cook("Hello @狮子")).to_not include("mention")
end
end
context "with Unicode usernames enabled" do
before { SiteSetting.unicode_usernames = true }
it "does detect mention" do
expect(
PrettyText.cook("Hello @狮子"),
).to match_html '<p>Hello <span class="mention">@狮子</span></p>'
end
end
DEV: Added modifier to change mentions extracted from cooked text (#21654) Added a new modifier hook to allow plugins to modify the @mentions extracted from a cooked text. Use case: Some plugins may change how the mentions are cooked to prevent them from being confused with user or group mentions and display the user card. This modifier hook allows the plugin to filter the mentions detected or add new ways to add mentions into cooked text.
2023-06-15 09:52:52 -04:00
context "with pretty_text_extract_mentions modifier" do
it "allows changing the mentions extracted" do
cooked_html = <<~HTML
<p>
<a class="mention" href="/u/test">@test</a>,
<a class="mention-group" href="/g/test-group">@test-group</a>,
<a class="custom-mention" href="/custom-mention">@test-custom</a>,
this is a test
</p>
HTML
extracted_mentions = PrettyText.extract_mentions(Nokogiri::HTML5.fragment(cooked_html))
expect(extracted_mentions).to include("test", "test-group")
expect(extracted_mentions).not_to include("test-custom")
Plugin::Instance
.new
.register_modifier(:pretty_text_extract_mentions) do |mentions, cooked_text|
custom_mentions =
cooked_text
.css(".custom-mention")
.map do |e|
if (name = e.inner_text)
name = name[1..-1]
name = User.normalize_username(name)
name
end
end
mentions + custom_mentions
end
extracted_mentions = PrettyText.extract_mentions(Nokogiri::HTML5.fragment(cooked_html))
expect(extracted_mentions).to include("test", "test-group", "test-custom")
ensure
DiscoursePluginRegistry.clear_modifiers!
end
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
describe "code fences" do
it "indents code correctly" do
code = <<~MD
X
```
#
x
```
MD
cooked = PrettyText.cook(code)
html = <<~HTML
<p>X</p>
<pre><code class="lang-auto"> #
x
</code></pre>
HTML
expect(cooked).to eq(html.strip)
end
it "doesn't replace emoji in code blocks with our emoji sets if emoji is enabled" do
expect(PrettyText.cook("```\n💣`\n```\n")).not_to match(/\:bomb\:/)
end
it "can include code class correctly" do
DEV: adds initial support for custom blocks using code fencing (#15743) Allows to write custom code blocks: ``` ```mermaid height=200,foo=bar test ``` ``` Which will then get converted to: ``` <pre data-code-wrap="mermaid" data-code-height="200" data-code-foo="bar"> <code class="lang-nohighlight"> test </code> </pre> ```
2022-02-09 05:23:44 -05:00
SiteSetting.highlighted_languages += "|c++|structured-text|p21"
Pass the full CommonMark spec
2017-07-21 13:20:45 -04:00
# keep in mind spaces should be trimmed per spec
expect(PrettyText.cook("``` ruby the mooby\n`````")).to eq(
'<pre><code class="lang-ruby"></code></pre>',
)
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(PrettyText.cook("```cpp\ncpp\n```")).to match_html(
"<pre><code class='lang-cpp'>cpp\n</code></pre>",
)
expect(PrettyText.cook("```\ncpp\n```")).to match_html(
"<pre><code class='lang-auto'>cpp\n</code></pre>",
FIX: Remove console warning for "nohighlight" (#19447)
2022-12-13 13:43:31 -05:00
)
expect(PrettyText.cook("```text\ncpp\n```")).to match_html(
"<pre><code class='lang-plaintext'>cpp\n</code></pre>",
)
expect(PrettyText.cook("```custom\ncustom content\n```")).to match_html(
"<pre data-code-wrap='custom'><code class='lang-plaintext'>custom content\n</code></pre>",
)
expect(PrettyText.cook("```custom foo=bar\ncustom content\n```")).to match_html(
"<pre data-code-foo='bar' data-code-wrap='custom'><code class='lang-plaintext'>custom content</code></pre>",
)
expect(PrettyText.cook("```INVALID a=1\n```")).to match_html(
"<pre data-code-a='1' data-code-wrap='INVALID'><code class='lang-plaintext'>\n</code></pre>",
)
expect(PrettyText.cook("```INVALID a=1, foo=bar , baz=2\n```")).to match_html(
"<pre data-code-a='1' data-code-foo='bar' data-code-baz='2' data-code-wrap='INVALID'><code class='lang-plaintext'>\n</code></pre>",
)
expect(PrettyText.cook("```text\n```")).to match_html(
"<pre><code class='lang-plaintext'>\n</code></pre>",
DEV: adds initial support for custom blocks using code fencing (#15743) Allows to write custom code blocks: ``` ```mermaid height=200,foo=bar test ``` ``` Which will then get converted to: ``` <pre data-code-wrap="mermaid" data-code-height="200" data-code-foo="bar"> <code class="lang-nohighlight"> test </code> </pre> ```
2022-02-09 05:23:44 -05:00
)
expect(PrettyText.cook("```auto\n```")).to match_html(
"<pre><code class='lang-auto'>\n</code></pre>",
)
expect(PrettyText.cook("```ruby startline=3 $%@#\n```")).to match_html(
"<pre data-code-startline='3'><code class='lang-ruby'>\n</code></pre>",
FIX: Remove console warning for "nohighlight" (#19447)
2022-12-13 13:43:31 -05:00
)
expect(PrettyText.cook("```mermaid a_-你=17\n```")).to match_html(
"<pre data-code-a_-='17' data-code-wrap='mermaid'><code class='lang-plaintext'>\n</code></pre>",
)
expect(
PrettyText.cook("```mermaid foo=<script>alert(document.cookie)</script>\n```"),
).to match_html(
"<pre data-code-foo='&lt;script&gt;alert(document.cookie)&lt;/script&gt;' data-code-wrap='mermaid'><code class='lang-plaintext'>\n</code></pre>",
)
expect(PrettyText.cook("```mermaid foo=‮ begin admin o\n```")).to match_html(
"<pre data-code-wrap='mermaid'><code class='lang-plaintext'>\n</code></pre>",
DEV: adds initial support for custom blocks using code fencing (#15743) Allows to write custom code blocks: ``` ```mermaid height=200,foo=bar test ``` ``` Which will then get converted to: ``` <pre data-code-wrap="mermaid" data-code-height="200" data-code-foo="bar"> <code class="lang-nohighlight"> test </code> </pre> ```
2022-02-09 05:23:44 -05:00
)
expect(PrettyText.cook("```c++\nc++\n```")).to match_html(
"<pre><code class='lang-c++'>c++\n</code></pre>",
)
expect(PrettyText.cook("```structured-text\nstructured-text\n```")).to match_html(
"<pre><code class='lang-structured-text'>structured-text\n</code></pre>",
)
expect(PrettyText.cook("```p21\np21\n```")).to match_html(
"<pre><code class='lang-p21'>p21\n</code></pre>",
)
expect(
PrettyText.cook("<pre data-code='3' data-code-foo='1' data-malicous-code='2'></pre>"),
).to match_html("<pre data-code-foo='1'></pre>")
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "indents code correctly" do
code = "X\n```\n\n #\n x\n```"
cooked = PrettyText.cook(code)
expect(cooked).to match_html(
"<p>X</p>\n<pre><code class=\"lang-auto\">\n #\n x\n</code></pre>",
)
Pending spec for Nokogiri fix
2014-10-02 22:37:07 -04:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "does censor code fences" do
fix intermittent failing tests, some watched word refactoring
2017-07-27 12:26:55 -04:00
begin
%w[apple banana].each do |w|
Fabricate(:watched_word, word: w, action: WatchedWord.actions[:censor])
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
end
fix intermittent failing tests, some watched word refactoring
2017-07-27 12:26:55 -04:00
expect(PrettyText.cook("# banana")).not_to include("banana")
ensure
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse"" This reverts commit 20780a1eeed56b321daf18ee6bbfe681a51d1bf4. * SECURITY: re-adds accidentally reverted commit: 03d26cd6: ensure embed_url contains valid http(s) uri * when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Discourse.redis.flushdb
fix intermittent failing tests, some watched word refactoring
2017-07-27 12:26:55 -04:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
SECURITY: Strip unrendered unicode bidirectional chars in code blocks (#15032) When rendering the markdown code blocks we replace the offending characters in the output string with spans highlighting a textual representation of the character, along with a title attribute with information about why the character was highlighted. The list of characters stripped by this fix, which are the bidirectional characters considered relevant, are: U+202A U+202B U+202C U+202D U+202E U+2066 U+2067 U+2068 U+2069
2021-11-21 19:43:03 -05:00
it "strips out unicode bidirectional (bidi) override characters and replaces with a highlighted span" do
code = <<~MD
X
DEV: adds initial support for custom blocks using code fencing (#15743) Allows to write custom code blocks: ``` ```mermaid height=200,foo=bar test ``` ``` Which will then get converted to: ``` <pre data-code-wrap="mermaid" data-code-height="200" data-code-foo="bar"> <code class="lang-nohighlight"> test </code> </pre> ```
2022-02-09 05:23:44 -05:00
```auto
SECURITY: Strip unrendered unicode bidirectional chars in code blocks (#15032) When rendering the markdown code blocks we replace the offending characters in the output string with spans highlighting a textual representation of the character, along with a title attribute with information about why the character was highlighted. The list of characters stripped by this fix, which are the bidirectional characters considered relevant, are: U+202A U+202B U+202C U+202D U+202E U+2066 U+2067 U+2068 U+2069
2021-11-21 19:43:03 -05:00
var isAdmin = false;
/*‮ begin admin only */⁦ if (isAdmin) ⁩ ⁦ {
console.log("You are an admin.");
/* end admins only ‮*/⁦ }
```
MD
cooked = PrettyText.cook(code)
hidden_bidi_title = I18n.t("post.hidden_bidi_character")
html = <<~HTML
<p>X</p>
<pre><code class="lang-auto">var isAdmin = false;
/*<span class="bidi-warning" title="#{hidden_bidi_title}">&lt;U+202E&gt;</span> begin admin only */<span class="bidi-warning" title="#{hidden_bidi_title}">&lt;U+2066&gt;</span> if (isAdmin) <span class="bidi-warning" title="#{hidden_bidi_title}">&lt;U+2069&gt;</span> <span class="bidi-warning" title="#{hidden_bidi_title}">&lt;U+2066&gt;</span> {
console.log("You are an admin.");
/* end admins only <span class="bidi-warning" title="#{hidden_bidi_title}">&lt;U+202E&gt;</span>*/<span class="bidi-warning" title="#{hidden_bidi_title}">&lt;U+2066&gt;</span> }
</code></pre>
HTML
expect(cooked).to eq(html.strip)
end
it "fuzzes all possible dangerous unicode bidirectional (bidi) override characters, making sure they are replaced" do
bad_bidi = [
"\u202A",
"\u202B",
"\u202C",
"\u202D",
"\u202E",
"\u2066",
"\u2067",
"\u2068",
"\u2069",
]
bad_bidi.each do |bidi|
code = <<~MD
```
#{bidi}
```
MD
cooked = PrettyText.cook(code)
formatted_bidi = format("&lt;U+%04X&gt;", bidi.ord)
html = <<~HTML
<pre><code class="lang-auto"><span class="bidi-warning" title="#{I18n.t("post.hidden_bidi_character")}">#{formatted_bidi}</span>
</code></pre>
HTML
expect(cooked).to eq(html.strip)
end
end
it "fuzzes all possible dangerous unicode bidirectional (bidi) override characters in solo code and pre nodes, making sure they are replaced" do
bad_bidi = [
"\u202A",
"\u202B",
"\u202C",
"\u202D",
"\u202E",
"\u2066",
"\u2067",
"\u2068",
"\u2069",
]
bad_bidi.each do |bidi|
code = <<~MD
<code>#{bidi}</code>
MD
cooked = PrettyText.cook(code)
formatted_bidi = format("&lt;U+%04X&gt;", bidi.ord)
html = <<~HTML
<p><code><span class="bidi-warning" title="#{I18n.t("post.hidden_bidi_character")}">#{formatted_bidi}</span></code></p>
HTML
expect(cooked).to eq(html.strip)
end
bad_bidi.each do |bidi|
code = <<~MD
<pre>#{bidi}</pre>
MD
cooked = PrettyText.cook(code)
formatted_bidi = format("&lt;U+%04X&gt;", bidi.ord)
html = <<~HTML
<pre><span class="bidi-warning" title="#{I18n.t("post.hidden_bidi_character")}">#{formatted_bidi}</span></pre>
HTML
expect(cooked).to eq(html.strip)
end
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
describe "rel attributes" do
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
before do
Nuke all `SiteSetting.stubs` from our codebase.
2017-07-07 02:09:14 -04:00
SiteSetting.add_rel_nofollow_to_user_content = true
SiteSetting.exclude_rel_nofollow_domains = "foo.com|bar.com"
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
2013-02-10 19:43:07 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should inject nofollow in all user provided links" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(PrettyText.cook('<a href="http://cnn.com">cnn</a>')).to match(/noopener nofollow ugc/)
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
2013-02-10 19:43:07 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should not inject nofollow in all local links" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(
PrettyText.cook("<a href='#{Discourse.base_url}/test.html'>cnn</a>") !~ /nofollow ugc/,
).to eq(true)
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
2013-02-10 19:43:07 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should not inject nofollow in all subdomain links" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(
PrettyText.cook(
"<a href='#{Discourse.base_url.sub("http://", "http://bla.")}/test.html'>cnn</a>",
) !~ /nofollow ugc/,
).to eq(true)
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
2013-02-10 19:43:07 -05:00
end
setting to exclude rel nofollow from particular domains
2013-02-11 02:58:19 -05:00
FIX: no-follow not handled correctly for sub domains if a.com was whitelisted aa.com would pass through
2015-05-27 00:31:01 -04:00
it "should inject nofollow in all non subdomain links" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(
PrettyText.cook(
"<a href='#{Discourse.base_url.sub("http://", "http://bla")}/test.html'>cnn</a>",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
).to match(/nofollow ugc/)
FIX: no-follow not handled correctly for sub domains if a.com was whitelisted aa.com would pass through
2015-05-27 00:31:01 -04:00
end
setting to exclude rel nofollow from particular domains
2013-02-11 02:58:19 -05:00
it "should not inject nofollow for foo.com" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(PrettyText.cook("<a href='http://foo.com/test.html'>cnn</a>") !~ /nofollow ugc/).to eq(
true,
)
setting to exclude rel nofollow from particular domains
2013-02-11 02:58:19 -05:00
end
FIX: no-follow not handled correctly for sub domains if a.com was whitelisted aa.com would pass through
2015-05-27 00:31:01 -04:00
it "should inject nofollow for afoo.com" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(PrettyText.cook("<a href='http://afoo.com/test.html'>cnn</a>")).to match(
/nofollow ugc/,
)
FIX: no-follow not handled correctly for sub domains if a.com was whitelisted aa.com would pass through
2015-05-27 00:31:01 -04:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
setting to exclude rel nofollow from particular domains
2013-02-11 02:58:19 -05:00
it "should not inject nofollow for bar.foo.com" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(
PrettyText.cook("<a href='http://bar.foo.com/test.html'>cnn</a>") !~ /nofollow ugc/,
).to eq(true)
setting to exclude rel nofollow from particular domains
2013-02-11 02:58:19 -05:00
end
Posts by trust level 3 users do not have nofollow on their external links.
2014-01-15 11:34:17 -05:00
it "should not inject nofollow if omit_nofollow option is given" do
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(
PrettyText.cook('<a href="http://cnn.com">cnn</a>', omit_nofollow: true) !~ /nofollow ugc/,
).to eq(true)
end
it "adds the noopener attribute even if omit_nofollow option is given" do
raw_html = '<a href="https://www.mysite.com/" target="_blank">Check out my site!</a>'
expect(PrettyText.cook(raw_html, omit_nofollow: true)).to match(/noopener/)
end
it "adds the noopener attribute even if omit_nofollow option is given" do
raw_html = '<a href="https://www.mysite.com/" target="_blank">Check out my site!</a>'
expect(PrettyText.cook(raw_html, omit_nofollow: false)).to match(/noopener nofollow ugc/)
Posts by trust level 3 users do not have nofollow on their external links.
2014-01-15 11:34:17 -05:00
end
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
2013-02-10 19:43:07 -05:00
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
describe "Excerpt" do
SECURITY: fix XSS in excerpt parser
2014-07-24 22:15:43 -04:00
it "sanitizes attempts to inject invalid attributes" do
spinner = "<a href=\"http://thedailywtf.com/\" data-bbcode=\"' class='fa fa-spin\">WTF</a>"
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt(spinner, 20)).to match_html spinner
SECURITY: fix XSS in excerpt parser
2014-07-24 22:15:43 -04:00
spinner =
%q{<a href="http://thedailywtf.com/" title="' class=&quot;fa fa-spin&quot;&gt;&lt;img src='http://thedailywtf.com/Resources/Images/Primary/logo.gif"></a>}
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt(spinner, 20)).to match_html spinner
SECURITY: fix XSS in excerpt parser
2014-07-24 22:15:43 -04:00
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with images" do
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
it "should dump images" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("<img src='http://cnn.com/a.gif'>", 100)).to eq("[image]")
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
end
option to strip links from excerpts
2013-04-29 23:25:55 -04:00
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with alt tags" do
FIX: Handle cases where `alt` and `title` tag is blank when parsing excerpt.
2017-04-11 00:12:51 -04:00
it "should keep alt tags" do
expect(
PrettyText.excerpt(
"<img src='http://cnn.com/a.gif' alt='car' title='my big car'>",
100,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
FIX: Handle cases where `alt` and `title` tag is blank when parsing excerpt.
2017-04-11 00:12:51 -04:00
).to eq("[car]")
end
describe "when alt tag is empty" do
it "should not keep alt tags" do
expect(PrettyText.excerpt("<img src='http://cnn.com/a.gif' alt>", 100)).to eq(
"[#{I18n.t("excerpt_image")}]",
)
end
end
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with title tags" do
FIX: Handle cases where `alt` and `title` tag is blank when parsing excerpt.
2017-04-11 00:12:51 -04:00
it "should keep title tags" do
expect(PrettyText.excerpt("<img src='http://cnn.com/a.gif' title='car'>", 100)).to eq(
"[car]",
)
end
describe "when title tag is empty" do
it "should not keep title tags" do
expect(PrettyText.excerpt("<img src='http://cnn.com/a.gif' title>", 100)).to eq(
"[#{I18n.t("excerpt_image")}]",
)
end
end
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
end
it "should convert images to markdown if the option is set" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt(
"<img src='http://cnn.com/a.gif' title='car'>",
100,
markdown_images: true,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
).to eq("![car](http://cnn.com/a.gif)")
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
FIX: Don't disable details when below truncate limit
2017-12-20 15:44:36 -05:00
it "should keep details if too long" do
FIX: Calculation of text length for <details> in excerpt was wrong
2018-03-12 11:52:06 -04:00
expect(
PrettyText.excerpt("<details><summary>expand</summary><p>hello</p></details>", 6),
).to match_html "<details class='disabled'><summary>expand</summary></details>"
FIX: details tags broke excerpts
2017-12-19 17:28:55 -05:00
end
FIX: Don't disable details when below truncate limit
2017-12-20 15:44:36 -05:00
it "doesn't disable details if short enough" do
expect(
PrettyText.excerpt("<details><summary>expand</summary><p>hello</p></details>", 60),
).to match_html "<details><summary>expand</summary>hello</details>"
end
DEV: Always `await` for `publishToMessageBus` (#17626) (+ typos)
2022-07-24 13:55:58 -04:00
it "should remove meta information" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt(wrapped_image, 100),
).to match_html "<a href='//localhost:3000/uploads/default/4399/33691397e78b4d75.png' class='lightbox' title='Screen Shot 2014-04-14 at 9.47.10 PM.png'>[image]</a>"
FIX: remove meta data from lightbox in both excerpt (html & text)
2014-11-05 14:37:00 -05:00
end
FEATURE: omit images from og and twitter description tags
2017-11-28 06:27:43 -05:00
it "should strip images when option is set" do
expect(
PrettyText.excerpt("<img src='http://cnn.com/a.gif'>", 100, strip_images: true),
).to be_blank
expect(
PrettyText.excerpt(
"<img src='http://cnn.com/a.gif'> Hello world!",
100,
strip_images: true,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
FEATURE: omit images from og and twitter description tags
2017-11-28 06:27:43 -05:00
).to eq("Hello world!")
end
it "should strip images, but keep emojis when option is set" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
emoji_image =
"<img src='/images/emoji/twitter/heart.png?v=#{Emoji::EMOJI_VERSION}' title=':heart:' class='emoji' alt=':heart:' loading='lazy' width='20' height='20'>"
FEATURE: omit images from og and twitter description tags
2017-11-28 06:27:43 -05:00
html = "<img src='http://cnn.com/a.gif'> Hello world #{emoji_image}"
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
expect(PrettyText.excerpt(html, 100, strip_images: true)).to eq("Hello world :heart:")
FEATURE: omit images from og and twitter description tags
2017-11-28 06:27:43 -05:00
expect(
PrettyText.excerpt(html, 100, strip_images: true, keep_emoji_images: true),
).to match_html("Hello world #{emoji_image}")
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with emojis" do
FIX: broken emojis in topic excerpt (#11667) When a post is truncated into the excerpt, sometimes we are breaking code of emoji
2021-01-10 18:40:41 -05:00
it "should remove broken emoji" do
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
html = <<~HTML
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<img src=\"//localhost:3000/images/emoji/twitter/bike.png?v=#{Emoji::EMOJI_VERSION}\" title=\":bike:\" class=\"emoji\" alt=\":bike:\" loading=\"lazy\" width=\"20\" height=\"20\"> <img src=\"//localhost:3000/images/emoji/twitter/cat.png?v=#{Emoji::EMOJI_VERSION}\" title=\":cat:\" class=\"emoji\" alt=\":cat:\" loading=\"lazy\" width=\"20\" height=\"20\"> <img src=\"//localhost:3000/images/emoji/twitter/discourse.png?v=#{Emoji::EMOJI_VERSION}\" title=\":discourse:\" class=\"emoji\" alt=\":discourse:\" loading=\"lazy\" width=\"20\" height=\"20\">
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
HTML
FIX: always truncate uncomplete emojis in excerpts (#11678) Additional fix after https://github.com/discourse/discourse/pull/11667 Always truncate "broken" emojis from excerpts.
2021-01-10 21:43:11 -05:00
expect(PrettyText.excerpt(html, 7)).to eq(":bike: &hellip;")
expect(PrettyText.excerpt(html, 8)).to eq(":bike: &hellip;")
expect(PrettyText.excerpt(html, 9)).to eq(":bike: &hellip;")
expect(PrettyText.excerpt(html, 10)).to eq(":bike: &hellip;")
expect(PrettyText.excerpt(html, 11)).to eq(":bike: &hellip;")
expect(PrettyText.excerpt(html, 12)).to eq(":bike: :cat: &hellip;")
expect(PrettyText.excerpt(html, 13)).to eq(":bike: :cat: &hellip;")
expect(PrettyText.excerpt(html, 14)).to eq(":bike: :cat: &hellip;")
FIX: broken emojis in topic excerpt (#11667) When a post is truncated into the excerpt, sometimes we are breaking code of emoji
2021-01-10 18:40:41 -05:00
end
end
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
it "should have an option to strip links" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("<a href='http://cnn.com'>cnn</a>", 100, strip_links: true)).to eq(
"cnn",
)
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
Allow images in the daily digest for top scoring posts
2013-06-05 18:54:46 -04:00
it "should preserve links" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt("<a href='http://cnn.com'>cnn</a>", 100),
).to match_html "<a href='http://cnn.com'>cnn</a>"
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should deal with special keys properly" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("<pre><b></pre>", 100)).to eq("")
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should truncate stuff properly" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("hello world", 5)).to eq("hello&hellip;")
expect(PrettyText.excerpt("<p>hello</p><p>world</p>", 6)).to eq("hello w&hellip;")
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should insert a space between to Ps" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("<p>a</p><p>b</p>", 5)).to eq("a b")
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should strip quotes" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("<aside class='quote'><p>a</p><p>b</p></aside>boom", 5)).to eq(
"boom",
)
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should not count the surrounds of a link" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt("<a href='http://cnn.com'>cnn</a>", 3),
).to match_html "<a href='http://cnn.com'>cnn</a>"
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
Better HTML emails, smarter email digests, new email section in admin with digest preview
2013-06-03 16:12:24 -04:00
it "uses an ellipsis instead of html entities if provided with the option" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt("<a href='http://cnn.com'>cnn</a>", 2, text_entities: true),
).to match_html "<a href='http://cnn.com'>cn...</a>"
Better HTML emails, smarter email digests, new email section in admin with digest preview
2013-06-03 16:12:24 -04:00
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should truncate links" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt("<a href='http://cnn.com'>cnn</a>", 2),
).to match_html "<a href='http://cnn.com'>cn&hellip;</a>"
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
FIX: Don't extract links from empty quotes
2014-05-20 17:20:52 -04:00
it "doesn't extract empty quotes as links" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.extract_links("<aside class='quote'>not a linked quote</aside>\n").to_a,
).to be_empty
FIX: Don't extract links from empty quotes
2014-05-20 17:20:52 -04:00
end
FIX: don't extract links from elided parts
2016-03-16 17:35:08 -04:00
it "doesn't extract links from elided parts" do
expect(
PrettyText.extract_links(
"<details class='elided'><a href='http://cnn.com'>cnn</a></details>\n",
).to_a,
).to be_empty
end
FEATURE: First Quote badge
2014-07-11 00:17:01 -04:00
def extract_urls(text)
PrettyText.extract_links(text).map(&:url).to_a
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should be able to extract links" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(extract_urls("<a href='http://cnn.com'>http://bla.com</a>")).to eq(["http://cnn.com"])
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
Fix bug where links to posts weren't being tracked
2013-02-13 15:22:04 -05:00
it "should extract links to topics" do
FIX: Don't add a slug to constructed quote urls (#12052) A topic with the slug 'topic' might exist and may end up being linked to by mistake when malformed (i.e. cross-site) quotes are posted.
2021-02-11 13:21:13 -05:00
expect(extract_urls("<aside class=\"quote\" data-topic=\"321\">aside</aside>")).to eq(
["/t/321"],
)
Fix bug where links to posts weren't being tracked
2013-02-13 15:22:04 -05:00
end
FIX: Improve top links section from user summary (#15675) * Do not extract links for hotlinked images * Include only links that have been clicked at least once in user summary
2022-01-23 19:33:23 -05:00
it "does not extract links from hotlinked images" do
html = <<~HTML
<p>
<a href="https://example.com">example</a>
<a href="https://images.pexels.com/photos/1525041/pexels-photo-1525041.jpeg?auto=compress&amp;cs=tinysrgb&amp;w=1260&amp;h=750&amp;dpr=2" target="_blank" rel="noopener" class="onebox">
<img src="https://images.pexels.com/photos/1525041/pexels-photo-1525041.jpeg?auto=compress&amp;cs=tinysrgb&amp;w=1260&amp;h=750&amp;dpr=2" width="690" height="459">
</a>
</p>
HTML
expect(extract_urls(html)).to eq(["https://example.com"])
end
DEV: Remove lazy-yt and replace with lazy-videos (#20722) - Refactors the old plugin to remove jquery usage - Adds support for Vimeo videos (default on) and Tiktok (experimental and default off)
2023-03-29 11:54:25 -04:00
context "when lazy-videos" do
it "should extract youtube url" do
expect(
extract_urls(
"<div class=\"lazy-video-container\" data-video-id=\"yXEuEUQIP3Q\" data-video-title=\"Mister Rogers defending PBS to the US Senate\" data-provider-name=\"youtube\"></div>",
),
).to eq(["https://www.youtube.com/watch?v=yXEuEUQIP3Q"])
end
it "should extract vimeo url" do
expect(
extract_urls(
"<div class=\"lazy-video-container\" data-video-id=\"786646692\" data-video-title=\"Dear Rich\" data-provider-name=\"vimeo\"></div>",
),
).to eq(["https://vimeo.com/786646692"])
end
it "should extract tiktok url" do
expect(
extract_urls(
"<div class=\"lazy-video-container\" data-video-id=\"6718335390845095173\" data-video-title=\"Scramble up ur name &amp;amp; I’ll try to guess it😍❤️ #foryoupage #petsoftiktok...\" data-provider-name=\"tiktok\"></div>",
),
).to eq(["https://m.tiktok.com/v/6718335390845095173"])
end
FIX: Consider lazyYT divs as links when extracting
2016-09-22 16:50:05 -04:00
end
Fix bug where links to posts weren't being tracked
2013-02-13 15:22:04 -05:00
it "should extract links to posts" do
FIX: Don't add a slug to constructed quote urls (#12052) A topic with the slug 'topic' might exist and may end up being linked to by mistake when malformed (i.e. cross-site) quotes are posted.
2021-02-11 13:21:13 -05:00
expect(
extract_urls("<aside class=\"quote\" data-topic=\"1234\" data-post=\"4567\">aside</aside>"),
).to eq(["/t/1234/4567"])
Fix bug where links to posts weren't being tracked
2013-02-13 15:22:04 -05:00
end
FIX: don't extract links to anchors
2016-04-15 14:02:18 -04:00
it "should not extract links to anchors" do
expect(extract_urls("<a href='#tos'>TOS</a>")).to eq([])
end
FIX: Have onebox ignore internal links
2013-06-05 14:53:07 -04:00
it "should not extract links inside quotes" do
FEATURE: First Quote badge
2014-07-11 00:17:01 -04:00
links =
PrettyText.extract_links(
"
FIX: Have onebox ignore internal links
2013-06-05 14:53:07 -04:00
<a href='http://body_only.com'>http://useless1.com</a>
<aside class=\"quote\" data-topic=\"1234\">
<a href='http://body_and_quote.com'>http://useless3.com</a>
<a href='http://quote_only.com'>http://useless4.com</a>
</aside>
<a href='http://body_and_quote.com'>http://useless2.com</a>
FEATURE: First Quote badge
2014-07-11 00:17:01 -04:00
",
)
FIX: wasn't extracting links to quoted posts
2017-02-06 08:45:04 -05:00
expect(links.map { |l| [l.url, l.is_quote] }.sort).to eq(
[
["http://body_only.com", false],
["http://body_and_quote.com", false],
FIX: Don't add a slug to constructed quote urls (#12052) A topic with the slug 'topic' might exist and may end up being linked to by mistake when malformed (i.e. cross-site) quotes are posted.
2021-02-11 13:21:13 -05:00
["/t/1234", true],
FIX: wasn't extracting links to quoted posts
2017-02-06 08:45:04 -05:00
].sort,
)
FIX: Have onebox ignore internal links
2013-06-05 14:53:07 -04:00
end
FIX: Do not check for duplicate links in Onebox (#13345) If a user posted a URL that appeared inside a Onebox, then the user got a duplicate link notice. This was fixed by skipping those links in Ruby. If a user posted a URL that was Oneboxes and contained other links that appeared in previous posts, then the user got a duplicate link notice. This was fixed by skipping those links in JavaScript.
2021-06-18 11:55:24 -04:00
it "should not extract links inside oneboxes" do
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
onebox = <<~HTML
FIX: Do not check for duplicate links in Onebox (#13345) If a user posted a URL that appeared inside a Onebox, then the user got a duplicate link notice. This was fixed by skipping those links in Ruby. If a user posted a URL that was Oneboxes and contained other links that appeared in previous posts, then the user got a duplicate link notice. This was fixed by skipping those links in JavaScript.
2021-06-18 11:55:24 -04:00
<aside class="onebox twitterstatus" data-onebox-src="https://twitter.com/EDBPostgres/status/1402528437441634306">
<header class="source">
<a href="https://twitter.com/EDBPostgres/status/1402528437441634306" target="_blank" rel="noopener">twitter.com</a>
<a href="https://twitter.com/EDBPostgres/status/1402528437441634306" target="_blank" rel="noopener">twitter.com</a>
</header>
<article class="onebox-body">
<div class="tweet">Example URL: <a target="_blank" href="https://example.com" rel="noopener">example.com</a></div>
</article>
</aside>
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
HTML
FIX: Do not check for duplicate links in Onebox (#13345) If a user posted a URL that appeared inside a Onebox, then the user got a duplicate link notice. This was fixed by skipping those links in Ruby. If a user posted a URL that was Oneboxes and contained other links that appeared in previous posts, then the user got a duplicate link notice. This was fixed by skipping those links in JavaScript.
2021-06-18 11:55:24 -04:00
expect(PrettyText.extract_links(onebox).map(&:url)).to contain_exactly(
"https://twitter.com/EDBPostgres/status/1402528437441634306",
)
end
remove trailing whitespaces :heart:
2013-02-25 11:42:20 -05:00
it "should not preserve tags in code blocks" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(
PrettyText.excerpt(
"<pre><code class='handlebars'>&lt;h3&gt;Hours&lt;/h3&gt;</code></pre>",
100,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
).to eq("&lt;h3&gt;Hours&lt;/h3&gt;")
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
it "should handle nil" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt(nil, 100)).to eq("")
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
FIX: pinned topic excerpt is not properly truncated
2013-05-10 06:28:17 -04:00
FEATURE: allow for custom excerpt BBCODE This allows fidelity in controlling excerpt (text that shows up when you pin a topic or link to it externally): ``` I am some text [excerpt] This is some **custom** markdown that should be the excerpt [/excerpt] More text ``` Previous solution relied on DIVs, unfortunately DIVs do not play well, by design with mixing markdown unless you have a preceding newline eg: ``` <div class='hello'> this will be treated properly as markdown </div> ``` This extra newline is not desirable. I am also considering adding ``` [div class=excerpt] [/div] ``` This would offer lots of flexibility to themes and plugins that do not want the extra annoying newline.
2018-12-25 10:02:28 -05:00
it "handles custom bbcode excerpt" do
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
raw = <<~MD
FEATURE: allow for custom excerpt BBCODE This allows fidelity in controlling excerpt (text that shows up when you pin a topic or link to it externally): ``` I am some text [excerpt] This is some **custom** markdown that should be the excerpt [/excerpt] More text ``` Previous solution relied on DIVs, unfortunately DIVs do not play well, by design with mixing markdown unless you have a preceding newline eg: ``` <div class='hello'> this will be treated properly as markdown </div> ``` This extra newline is not desirable. I am also considering adding ``` [div class=excerpt] [/div] ``` This would offer lots of flexibility to themes and plugins that do not want the extra annoying newline.
2018-12-25 10:02:28 -05:00
[excerpt]
hello [site](https://site.com)
[/excerpt]
more stuff
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
MD
FEATURE: allow for custom excerpt BBCODE This allows fidelity in controlling excerpt (text that shows up when you pin a topic or link to it externally): ``` I am some text [excerpt] This is some **custom** markdown that should be the excerpt [/excerpt] More text ``` Previous solution relied on DIVs, unfortunately DIVs do not play well, by design with mixing markdown unless you have a preceding newline eg: ``` <div class='hello'> this will be treated properly as markdown </div> ``` This extra newline is not desirable. I am also considering adding ``` [div class=excerpt] [/div] ``` This would offer lots of flexibility to themes and plugins that do not want the extra annoying newline.
2018-12-25 10:02:28 -05:00
post = Fabricate(:post, raw: raw)
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expect(post.excerpt).to eq(
"hello <a href=\"https://site.com\" rel=\"noopener nofollow ugc\">site</a>",
)
FEATURE: allow for custom excerpt BBCODE This allows fidelity in controlling excerpt (text that shows up when you pin a topic or link to it externally): ``` I am some text [excerpt] This is some **custom** markdown that should be the excerpt [/excerpt] More text ``` Previous solution relied on DIVs, unfortunately DIVs do not play well, by design with mixing markdown unless you have a preceding newline eg: ``` <div class='hello'> this will be treated properly as markdown </div> ``` This extra newline is not desirable. I am also considering adding ``` [div class=excerpt] [/div] ``` This would offer lots of flexibility to themes and plugins that do not want the extra annoying newline.
2018-12-25 10:02:28 -05:00
end
FIX: Ignore max excerpt length for div excerpts too (#13058) We support two types of custom excerpts. It can be <div class="excerpt"> or <span class="excerpt">: https://github.com/discourse/discourse/blob/b21f74060e865d809ba466cb52e6cb95c7b0cf1f/lib/excerpt_parser.rb#L120 We also ignore max excerpt length for custom excerpts. But we forgot to process div when ignoring max length.
2021-05-24 05:05:24 -04:00
it "handles div excerpt at the beginning of a post" do
expect(PrettyText.excerpt("<div class='excerpt'>hi</div> test", 100)).to eq("hi")
end
FEATURE: Allow manual excerpt to be defined past the beginning of the post There is still a limitation that the span excerpt must begin before the post_excerpt_max_length.
2014-09-03 03:12:56 -04:00
it "handles span excerpt at the beginning of a post" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("<span class='excerpt'>hi</span> test", 100)).to eq("hi")
FIX: Ignore max excerpt length for div excerpts too (#13058) We support two types of custom excerpts. It can be <div class="excerpt"> or <span class="excerpt">: https://github.com/discourse/discourse/blob/b21f74060e865d809ba466cb52e6cb95c7b0cf1f/lib/excerpt_parser.rb#L120 We also ignore max excerpt length for custom excerpts. But we forgot to process div when ignoring max length.
2021-05-24 05:05:24 -04:00
end
it "ignores max excerpt length if a div excerpt is specified" do
two_hundred = "123456789 " * 20 + "."
text = two_hundred + "<div class='excerpt'>#{two_hundred}</div>" + two_hundred
expect(PrettyText.excerpt(text, 100)).to eq(two_hundred)
Feature: allow mods to cut pinned topic excerpts
2014-07-17 07:32:17 -04:00
end
FEATURE: Allow manual excerpt to be specified anywhere in the post and override max excerpt length
2014-09-04 01:03:12 -04:00
it "ignores max excerpt length if a span excerpt is specified" do
two_hundred = "123456789 " * 20 + "."
text = two_hundred + "<span class='excerpt'>#{two_hundred}</span>" + two_hundred
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt(text, 100)).to eq(two_hundred)
FEATURE: Allow manual excerpt to be defined past the beginning of the post There is still a limitation that the span excerpt must begin before the post_excerpt_max_length.
2014-09-03 03:12:56 -04:00
end
FIX: properly unescape HTML entities in excerpts
2014-12-10 06:52:51 -05:00
it "unescapes html entities when we want text entities" do
few components with rspec3 syntax
2015-01-09 11:34:37 -05:00
expect(PrettyText.excerpt("&#39;", 500, text_entities: true)).to eq("'")
FIX: properly unescape HTML entities in excerpts
2014-12-10 06:52:51 -05:00
end
FIX: Emoji in Discourse onebox is wrapped in square brackets.
2015-12-14 08:46:15 -05:00
it "should have an option to preserve emoji images" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
emoji_image =
"<img src='/images/emoji/twitter/heart.png?v=#{Emoji::EMOJI_VERSION}' title=':heart:' class='emoji' alt=':heart:' loading='lazy' width='20' height='20'>"
FIX: Emoji in Discourse onebox is wrapped in square brackets.
2015-12-14 08:46:15 -05:00
expect(PrettyText.excerpt(emoji_image, 100, keep_emoji_images: true)).to match_html(
emoji_image,
)
end
FEATURE: remap emojis back for push notifications and desktop alerts
2016-10-10 22:03:21 -04:00
it "should have an option to remap emoji to code points" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
emoji_image =
"I <img src='/images/emoji/twitter/heart.png?v=#{Emoji::EMOJI_VERSION}' title=':heart:' class='emoji' alt=':heart:' loading='lazy' width='20' height='20'> you <img src='/images/emoji/twitter/heart.png?v=#{Emoji::EMOJI_VERSION}' title=':unknown:' class='emoji' alt=':unknown:' loading='lazy' width='20' height='20'> "
FEATURE: remap emojis back for push notifications and desktop alerts
2016-10-10 22:03:21 -04:00
expect(PrettyText.excerpt(emoji_image, 100, remap_emoji: true)).to match_html(
"I ❤ you :unknown:",
)
end
FIX: Emoji in Discourse onebox is wrapped in square brackets.
2015-12-14 08:46:15 -05:00
it "should have an option to preserve emoji codes" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
emoji_code =
"<img src='/images/emoji/twitter/heart.png?v=#{Emoji::EMOJI_VERSION}' title=':heart:' class='emoji' alt=':heart:' loading='lazy' width='20' height='20'>"
UX: we should always simply use emoji codes as opposed to treating as image
2016-06-01 22:29:25 -04:00
expect(PrettyText.excerpt(emoji_code, 100)).to eq(":heart:")
FEATURE: Display emojis in user stream.
2015-07-23 11:02:03 -04:00
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
context "with option to preserve onebox source" do
FIX: Quotes should be ignored when parsing for onebox source.
2017-04-11 03:13:21 -04:00
it "should return the right excerpt" do
FIX: use allowlist and blocklist terminology (#10209) This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-26 20:23:54 -04:00
onebox =
"<aside class=\"onebox allowlistedgeneric\">\n <header class=\"source\">\n <a href=\"https://meta.discourse.org/t/infrequent-translation-updates-in-stable-branch/31213/9\">meta.discourse.org</a>\n </header>\n <article class=\"onebox-body\">\n <img src=\"https://cdn-enterprise.discourse.org/meta/user_avatar/meta.discourse.org/gerhard/200/70381_1.png\" width=\"\" height=\"\" class=\"thumbnail\">\n\n<h3><a href=\"https://meta.discourse.org/t/infrequent-translation-updates-in-stable-branch/31213/9\">Infrequent translation updates in stable branch</a></h3>\n\n<p>Well, there's an Italian translation for \"New Topic\" in beta, it's been there since November 2014 and it works here on meta. Do you have any plugins installed? Try disabling them. I'm quite confident that it's either a plugin or a site...</p>\n\n </article>\n <div class=\"onebox-metadata\">\n \n \n </div>\n <div style=\"clear: both\"></div>\n</aside>\n\n\n"
FIX: Quotes should be ignored when parsing for onebox source.
2017-04-11 03:13:21 -04:00
expected =
"<a href=\"https://meta.discourse.org/t/infrequent-translation-updates-in-stable-branch/31213/9\">meta.discourse.org</a>"
FEATURE: Add option for `ExcerptParser` to keep onebox source.
2017-04-10 04:11:58 -04:00
FIX: Quotes should be ignored when parsing for onebox source.
2017-04-11 03:13:21 -04:00
expect(PrettyText.excerpt(onebox, 100, keep_onebox_source: true)).to eq(expected)
FEATURE: Add option for `ExcerptParser` to keep onebox source.
2017-04-10 04:11:58 -04:00
FIX: Quotes should be ignored when parsing for onebox source.
2017-04-11 03:13:21 -04:00
expect(
PrettyText.excerpt("#{onebox}\n \n \n \n\n\n #{onebox}", 100, keep_onebox_source: true),
).to eq("#{expected}\n\n#{expected}")
end
it "should continue to strip quotes" do
expect(
PrettyText.excerpt(
"<aside class='quote'><p>a</p><p>b</p></aside>boom",
100,
keep_onebox_source: true,
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
FIX: Quotes should be ignored when parsing for onebox source.
2017-04-11 03:13:21 -04:00
).to eq("boom")
end
FEATURE: Add option for `ExcerptParser` to keep onebox source.
2017-04-10 04:11:58 -04:00
end
FIX: Strip audio/video content from excerpt (#8881)
2020-02-06 15:08:13 -05:00
it "should strip audio/video" do
html = <<~HTML
<audio controls>
<source src="https://awebsite.com/audio.mp3"><a href="https://awebsite.com/audio.mp3">https://awebsite.com/audio.mp3</a></source>
</audio>
<p>Listen to this!</p>
HTML
expect(PrettyText.excerpt(html, 100)).to eq("Listen to this!")
html = <<~HTML
<div class="onebox video-onebox">
<video controlslist="nodownload" width="100%" height="100%" controls="">
<source src="http://videosource.com/running.mp4">
<a href="http://videosource.com/running.mp4">http://videosource.com/running.mp4</a>
</video>
</div>
DEV: Improve video onebox stripping spec Followup to 70819080
2020-02-20 11:44:54 -05:00
<p>Watch this, but do not include the video in the excerpt.</p>
FIX: Strip audio/video content from excerpt (#8881)
2020-02-06 15:08:13 -05:00
HTML
DEV: Improve video onebox stripping spec Followup to 70819080
2020-02-20 11:44:54 -05:00
ellipsis = "&hellip;"
excerpt_size = 40
excerpt = PrettyText.excerpt(html, excerpt_size)
expect(excerpt.size).to eq(excerpt_size + ellipsis.size)
expect(excerpt).to eq("Watch this, but do not include the video#{ellipsis}")
FIX: Strip audio/video content from excerpt (#8881)
2020-02-06 15:08:13 -05:00
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end
FIX: Strip links from google indexed bios when the users are new.
2013-06-05 15:28:10 -04:00
describe "strip links" do
it "returns blank for blank input" do
expect(PrettyText.strip_links("")).to be_blank
end
it "does nothing to a string without links" do
expect(PrettyText.strip_links("I'm the <b>batman</b>")).to eq("I'm the <b>batman</b>")
end
it "strips links but leaves the text content" do
expect(
PrettyText.strip_links(
"I'm the linked <a href='http://en.wikipedia.org/wiki/Batman'>batman</a>",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
FIX: Strip links from google indexed bios when the users are new.
2013-06-05 15:28:10 -04:00
).to eq("I'm the linked batman")
end
SECURITY: Stripping links could unescape html fragments
2014-09-17 12:08:00 -04:00
it "escapes the text content" do
expect(
PrettyText.strip_links(
"I'm the linked <a href='http://en.wikipedia.org/wiki/Batman'>&lt;batman&gt;</a>",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
),
SECURITY: Stripping links could unescape html fragments
2014-09-17 12:08:00 -04:00
).to eq("I'm the linked &lt;batman&gt;")
end
FIX: Strip links from google indexed bios when the users are new.
2013-06-05 15:28:10 -04:00
end
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
2013-02-10 19:43:07 -05:00
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
describe "strip_image_wrapping" do
def strip_image_wrapping(html)
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-04 23:46:57 -04:00
doc = Nokogiri::HTML5.fragment(html)
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
described_class.strip_image_wrapping(doc)
FIX: Don't include image meta data when embedded in an email
2014-04-17 12:32:51 -04:00
doc.to_html
end
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
it "doesn't change HTML when there's no wrapped image" do
html = "<img src=\"wat.png\">"
expect(strip_image_wrapping(html)).to eq(html)
end
it "strips the metadata" do
expect(
strip_image_wrapping(wrapped_image),
).to match_html "<div class=\"lightbox-wrapper\"><a href=\"//localhost:3000/uploads/default/4399/33691397e78b4d75.png\" class=\"lightbox\" title=\"Screen Shot 2014-04-14 at 9.47.10 PM.png\"><img src=\"//localhost:3000/uploads/default/_optimized/bd9/b20/bbbcd6a0c0_655x500.png\" width=\"655\" height=\"500\"></a></div>"
end
end
describe "format_for_email" do
let(:base_url) { "http://baseurl.net" }
FIX: emails with embedded posts should always use absolute URLs
2013-11-28 15:57:21 -05:00
before { Discourse.stubs(:base_url).returns(base_url) }
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
it "does not crash" do
PrettyText.format_for_email(
'<a href="mailto:michael.brown@discourse.org?subject=Your%20post%20at%20http://try.discourse.org/t/discussion-happens-so-much/127/1000?u=supermathie">test</a>',
post,
)
end
FIX: emails with embedded posts should always use absolute URLs
2013-11-28 15:57:21 -05:00
it "adds base url to relative links" do
Convert server side paths to use `/u/`
2017-03-28 14:27:54 -04:00
html =
"<p><a class=\"mention\" href=\"/u/wiseguy\">@wiseguy</a>, <a class=\"mention\" href=\"/u/trollol\">@trollol</a> what do you guys think? </p>"
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
output = described_class.format_for_email(html, post)
Convert server side paths to use `/u/`
2017-03-28 14:27:54 -04:00
expect(output).to eq(
"<p><a class=\"mention\" href=\"#{base_url}/u/wiseguy\">@wiseguy</a>, <a class=\"mention\" href=\"#{base_url}/u/trollol\">@trollol</a> what do you guys think? </p>",
)
FIX: emails with embedded posts should always use absolute URLs
2013-11-28 15:57:21 -05:00
end
it "doesn't change external absolute links" do
html = "<p>Check out <a href=\"http://mywebsite.com/users/boss\">this guy</a>.</p>"
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
expect(described_class.format_for_email(html, post)).to eq(html)
FIX: emails with embedded posts should always use absolute URLs
2013-11-28 15:57:21 -05:00
end
it "doesn't change internal absolute links" do
html = "<p>Check out <a href=\"#{base_url}/users/boss\">this guy</a>.</p>"
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
expect(described_class.format_for_email(html, post)).to eq(html)
FIX: emails with embedded posts should always use absolute URLs
2013-11-28 15:57:21 -05:00
end
it "can tolerate invalid URLs" do
html = "<p>Check out <a href=\"not a real url\">this guy</a>.</p>"
FEATURE: More granular mailing list mode (#4068) * Rearrange frontend to account for mailing list mode * Allow update of user preference for mailing list frequency * Add mailing list frequency estimate * Simplify frequency estimate; disable activity summary for mailing list mode * Remove combined updates * Add specs for enqueue mailing list mode job * Write mailing list method for mailer * Fix linting error * Account for stale topics * Add translations for default mailing list setting * One query for mailing list topics * Fix failing spec * WIP * Flesh out html template * First pass at text-based mailing list summary * Add user avatar * Properly format posts for mailing list * Move make_all_links_absolute into Email::Styles * Apply first_seen_at to user * Send mailing list email summary hourly based on first_seen_at * Branch and test cleanup * Use existing mailing list mode estimate * Fix failing specs
2016-05-21 09:17:54 -04:00
expect { described_class.format_for_email(html, post) }.to_not raise_error
FIX: crash on invalid uri component
2014-07-30 03:09:55 -04:00
end
FIX: broken mailto href's in emails
2018-06-08 13:11:52 -04:00
it "doesn't change mailto" do
html = "<p>Contact me at <a href=\"mailto:username@me.com\">this address</a>.</p>"
expect(PrettyText.format_for_email(html, post)).to eq(html)
end
FIX: escape Vimeo iframe URL
2019-05-09 11:37:55 -04:00
it "prefers data-original-href attribute to get Vimeo iframe link and escapes it" do
html =
"<p>Check out this video – <iframe src='https://player.vimeo.com/video/329875646' data-original-href='https://vimeo.com/329875646/> <script>alert(1)</script>'></iframe>.</p>"
expect(PrettyText.format_for_email(html, post)).to match(
Regexp.escape("https://vimeo.com/329875646/%3E%20%3Cscript%3Ealert(1)%3C/script%3E"),
)
end
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
FIX: vimeo iframe url when data-original-href is missing (#18894)
2023-01-31 06:00:27 -05:00
it "creates a valid URL when data-original-href is missing from Vimeo link" do
html =
'<iframe src="https://player.vimeo.com/video/508864124?h=fcbbcc92fa" width="640" height="360" frameborder="0" allow="autoplay; fullscreen; picture-in-picture" allowfullscreen></iframe>'
expect(PrettyText.format_for_email(html, post)).to match(
"https://vimeo.com/508864124/fcbbcc92fa",
)
end
FIX: Replace Vimeo iframes with a link in emails (#11443) This was implemented before, but it was not tested and broke at some point (probably Nokogiri update).
2020-12-09 07:58:36 -05:00
describe "#convert_vimeo_iframes" do
it "converts <iframe> to <a>" do
html = <<~HTML
<p>This is a Vimeo link:</p>
<iframe width="640" height="360" src="https://player.vimeo.com/video/1" data-original-href="https://vimeo.com/1" frameborder="0" allowfullscreen="" seamless="seamless" sandbox="allow-same-origin allow-scripts allow-forms allow-popups allow-popups-to-escape-sandbox allow-presentation"></iframe>
HTML
md = PrettyText.format_for_email(html, post)
expect(md).not_to include("<iframe")
expect(md).to match_html(<<~HTML)
<p>This is a Vimeo link:</p>
<p><a href="https://vimeo.com/1">https://vimeo.com/1</a></p>
HTML
end
end
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
describe "#strip_secure_uploads" do
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
before do
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 07:32:25 -04:00
setup_s3
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
SiteSetting.s3_cdn_url = "https://s3.cdn.com"
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
SiteSetting.secure_uploads = true
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
SiteSetting.login_required = true
end
it "replaces secure video content" do
html = <<~HTML
<video width="100%" height="100%" controls="">
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
<source src="#{base_url}/secure-uploads/original/1X/some-video.mp4">
<a href="#{base_url}/secure-uploads/original/1X/some-video.mp4">Video label</a>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
</source>
</video>
HTML
md = PrettyText.format_for_email(html, post)
expect(md).not_to include("<video")
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
expect(md.to_s).to match(I18n.t("emails.secure_uploads_placeholder"))
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
expect(md.to_s).not_to match(SiteSetting.Upload.s3_cdn_url)
end
it "replaces secure audio content" do
html = <<~HTML
<audio controls>
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
<source src="#{base_url}/secure-uploads/original/1X/some-audio.mp3">
<a href="#{base_url}/secure-uploads/original/1X/some-audio.mp3">Audio label</a>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
</source>
</audio>
HTML
md = PrettyText.format_for_email(html, post)
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
expect(md).not_to include("<audio")
expect(md.to_s).to match(I18n.t("emails.secure_uploads_placeholder"))
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
expect(md.to_s).not_to match(SiteSetting.Upload.s3_cdn_url)
end
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
it "replaces secure uploads within a link with a placeholder, keeping the url in an attribute" do
url = "#{Discourse.base_url}\/secure-uploads/original/1X/testimage.png"
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
html = <<~HTML
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
<a href=\"#{url}\"><img src=\"/secure-uploads/original/1X/testimage.png\"></a>
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
HTML
md = PrettyText.format_for_email(html, post)
expect(md).not_to include("<img")
expect(md).to include("Redacted")
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
expect(md).to include("data-stripped-secure-upload=\"#{url}\"")
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
end
it "does not create nested redactions from double processing because of the view media link" do
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
url = "#{Discourse.base_url}\/secure-uploads/original/1X/testimage.png"
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
html = <<~HTML
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
<a href=\"#{url}\"><img src=\"/secure-uploads/original/1X/testimage.png\"></a>
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
HTML
md = PrettyText.format_for_email(html, post)
md = PrettyText.format_for_email(md, post)
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
expect(md.scan(/stripped-secure-view-upload/).length).to eq(1)
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
expect(md.scan(/Redacted/).length).to eq(1)
end
it "replaces secure images with a placeholder, keeping the url in an attribute" do
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
url = "/secure-uploads/original/1X/testimage.png"
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
html = <<~HTML
FIX: persist secure image width and height if is given (#10994) `max-width: 50%; max-height: 400px;` is a good fallback, however, if width and height are given and are smaller than fallback - we should persist that smaller size.
2020-10-21 22:25:09 -04:00
<img src=\"#{url}\" width=\"20\" height=\"20\">
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
HTML
md = PrettyText.format_for_email(html, post)
expect(md).not_to include("<img")
expect(md).to include("Redacted")
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-28 19:24:33 -04:00
expect(md).to include("data-stripped-secure-upload=\"#{url}\"")
FIX: persist secure image width and height if is given (#10994) `max-width: 50%; max-height: 400px;` is a good fallback, however, if width and height are given and are smaller than fallback - we should persist that smaller size.
2020-10-21 22:25:09 -04:00
expect(md).to include("data-width=\"20\"")
expect(md).to include("data-height=\"20\"")
FEATURE: Allow email image embed with secure media (#10563) This PR introduces a few important changes to secure media redaction in emails. First of all, two new site settings have been introduced: * `secure_media_allow_embed_images_in_emails`: If enabled we will embed secure images in emails instead of redacting them. * `secure_media_max_email_embed_image_size_kb`: The cap to the size of the secure image we will embed, defaulting to 1mb, so the email does not become too big. Max is 10mb. Works in tandem with `email_total_attachment_size_limit_kb`. `Email::Sender` will now attach images to the email based on these settings. The sender will also call `inline_secure_images` in `Email::Styles` after secure media is redacted and attachments are added to replace redaction messages with attached images. I went with attachment and `cid` URLs because base64 image support is _still_ flaky in email clients. All redaction of secure media is now handled in `Email::Styles` and calls out to `PrettyText.strip_secure_media` to do the actual stripping and replacing with placeholders. `app/mailers/group_smtp_mailer.rb` and `app/mailers/user_notifications.rb` no longer do any stripping because they are earlier in the pipeline than `Email::Styles`. Finally the redaction notice has been restyled and includes a link to the media that the user can click, which will show it to them if they have the necessary permissions. ![image](https://user-images.githubusercontent.com/920448/92341012-b9a2c380-f0ff-11ea-860e-b376b4528357.png)
2020-09-09 19:50:16 -04:00
end
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-17 20:25:42 -05:00
end
FIX: crash on invalid uri component
2014-07-30 03:09:55 -04:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "Is smart about linebreaks and IMG tags" do
raw = <<~MD
a <img>
<img>
<img>
<img>
<img>
a
<img>
- li
<img>
```
test
```
```
test
```
MD
html = <<~HTML
<p>a <img><br>
<img></p>
correct more edge cases with new engine
2017-07-17 17:25:40 -04:00
<p><img><br>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<img></p>
correct more edge cases with new engine
2017-07-17 17:25:40 -04:00
<p><img></p>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<p>a</p>
correct more edge cases with new engine
2017-07-17 17:25:40 -04:00
<p><img></p>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<ul>
<li>li</li>
</ul>
correct more edge cases with new engine
2017-07-17 17:25:40 -04:00
<p><img></p>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<pre><code class="lang-auto">test
</code></pre>
<pre><code class="lang-auto">test
</code></pre>
HTML
expect(PrettyText.cook(raw)).to eq(html.strip)
FEATURE: Allow selection of highlight js languages PERF: stop loading highlight js on load To get latest highlight js run bin/rake highlightjs:update
2015-03-13 01:15:13 -04:00
end
FEATURE: Perform a server side replacement of unicode emoji
2015-12-29 16:27:56 -05:00
describe "emoji" do
it "replaces unicode emoji with our emoji sets if emoji is enabled" do
expect(PrettyText.cook("💣")).to match(/\:bomb\:/)
end
FEATURE: site setting to allow html tables (which may come from imports) (allow_html_tables , disabled by default)
2015-07-20 02:56:32 -04:00
FEATURE: do not replace &harr; with an emoji &harr; technically can be an emoji, but usually will not be used in that way
2019-08-30 01:06:23 -04:00
it "does not replace left right arrow" do
expect(PrettyText.cook("&harr;")).to eq("<p>↔</p>")
end
FIX: Don't substitute emojis within code blocks
2016-03-02 14:31:32 -05:00
it "doesn't replace emoji in inline code blocks with our emoji sets if emoji is enabled" do
expect(PrettyText.cook("`💣`")).not_to match(/\:bomb\:/)
end
Add more server side unicode replacements for emoji
2015-12-30 14:46:52 -05:00
it "replaces some glyphs that are not in the emoji range" do
DEV: replaces huge generated emoji list by a simpler regex (#11053) Note that this commit is also fixing various mistakes in emojis. Some of them have been fixed manually in db.json/data.js/groups.json and will need to be fixed in emoji-db gem.
2021-04-22 02:43:06 -04:00
expect(PrettyText.cook("☹")).to match(/\:frowning\:/)
DEV: Update javascript:update_constants rake task following template colocation (#20365) As of https://github.com/discourse/discourse/commit/ba3f62f576068035bc8fbda615e595960cd4520c, handlebars templates are colocated with js files so the path to hbs templates referenced by this rake task is no longer valid. This commit fixes the path to hbs templates and updates a couple of files that are generated by the rake task.
2023-02-19 22:20:47 -05:00
expect(PrettyText.cook("☺")).to match(/\:smiling_face\:/)
DEV: replaces huge generated emoji list by a simpler regex (#11053) Note that this commit is also fixing various mistakes in emojis. Some of them have been fixed manually in db.json/data.js/groups.json and will need to be fixed in emoji-db gem.
2021-04-22 02:43:06 -04:00
expect(PrettyText.cook("☻")).to match(/\:slight_smile\:/)
expect(PrettyText.cook("♡")).to match(/\:heart\:/)
expect(PrettyText.cook("❤")).to match(/\:heart\:/)
expect(PrettyText.cook("❤️")).to match(/\:heart\:/) # in emoji range but ensure it works along others
Add more server side unicode replacements for emoji
2015-12-30 14:46:52 -05:00
end
FIX: allows replacement of digits and symbols emojis (#7978)
2019-08-07 05:38:58 -04:00
it "replaces digits" do
expect(PrettyText.cook("🔢")).to match(/\:1234\:/)
expect(PrettyText.cook("1️⃣")).to match(/\:one\:/)
expect(PrettyText.cook("#️⃣")).to match(/\:hash\:/)
expect(PrettyText.cook("*️⃣")).to match(/\:asterisk\:/)
end
FEATURE: Perform a server side replacement of unicode emoji
2015-12-29 16:27:56 -05:00
it "doesn't replace unicode emoji if emoji is disabled" do
FIX: Emoji in Discourse onebox is wrapped in square brackets.
2015-12-14 08:46:15 -05:00
SiteSetting.enable_emoji = false
FEATURE: Perform a server side replacement of unicode emoji
2015-12-29 16:27:56 -05:00
expect(PrettyText.cook("💣")).not_to match(/\:bomb\:/)
end
Transforms native skin toned emojis to valid names
2017-06-14 09:35:37 -04:00
FEATURE: allow site operators to disable emoji shortcuts
2018-01-23 20:21:44 -05:00
it "doesn't replace emoji if emoji is disabled" do
SiteSetting.enable_emoji = false
expect(PrettyText.cook(":bomb:")).to eq("<p>:bomb:</p>")
end
it "doesn't replace shortcuts if disabled" do
SiteSetting.enable_emoji_shortcuts = false
expect(PrettyText.cook(":)")).to eq("<p>:)</p>")
end
it "does replace shortcuts if enabled" do
expect(PrettyText.cook(":)")).to match("smile")
end
Transforms native skin toned emojis to valid names
2017-06-14 09:35:37 -04:00
it "replaces skin toned emoji" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
expect(PrettyText.cook("hello 👱🏿‍♀️")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/blonde_woman/6.png?v=#{Emoji::EMOJI_VERSION}\" title=\":blonde_woman:t6:\" class=\"emoji\" alt=\":blonde_woman:t6:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
expect(PrettyText.cook("hello 👩‍🎤")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/woman_singer.png?v=#{Emoji::EMOJI_VERSION}\" title=\":woman_singer:\" class=\"emoji\" alt=\":woman_singer:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
expect(PrettyText.cook("hello 👩🏾‍🎓")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/woman_student/5.png?v=#{Emoji::EMOJI_VERSION}\" title=\":woman_student:t5:\" class=\"emoji\" alt=\":woman_student:t5:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
expect(PrettyText.cook("hello 🤷‍♀️")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/woman_shrugging.png?v=#{Emoji::EMOJI_VERSION}\" title=\":woman_shrugging:\" class=\"emoji\" alt=\":woman_shrugging:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
Transforms native skin toned emojis to valid names
2017-06-14 09:35:37 -04:00
end
FIX: Safari issue with some emojis (#6456) https://meta.discourse.org/t/emojis-selected-on-ios-displaying-additional-rectangles/86132/8
2018-10-07 22:32:25 -04:00
it "correctly strips VARIATION SELECTOR-16 character (ufe0f) from some emojis" do
expect(PrettyText.cook("❤️💣")).to match(/<img src[^>]+bomb[^>]+>/)
end
FEATURE: Detect emoji from Emoji 14.0 (#16531)
2022-04-22 02:42:15 -04:00
it "replaces Emoji from Unicode 14.0" do
expect(PrettyText.cook("🫣")).to match(/\:face_with_peeking_eye\:/)
end
FIX: pretty-text shims - getURL's baseUri (#20822)
2023-03-30 08:35:06 -04:00
context "with subfolder" do
it "prepends the subfolder path to the emoji url" do
set_subfolder "/forum"
expected = "src=\"/forum/images/emoji/twitter/grinning.png?v=#{Emoji::EMOJI_VERSION}\""
expect(PrettyText.cook("😀")).to include(expected)
expect(PrettyText.cook(":grinning:")).to include(expected)
end
it "prepends the subfolder path even if it is part of the emoji url" do
set_subfolder "/info"
expected =
"src=\"/info/images/emoji/twitter/information_source.png?v=#{Emoji::EMOJI_VERSION}\""
expect(PrettyText.cook("ℹ️")).to include(expected)
expect(PrettyText.cook(":information_source:")).to include(expected)
end
end
FEATURE: site setting to allow html tables (which may come from imports) (allow_html_tables , disabled by default)
2015-07-20 02:56:32 -04:00
end
REFACTOR: Migrate markdown functionality in ES6
2016-06-14 14:31:51 -04:00
describe "custom emoji" do
it "replaces the custom emoji" do
FIX: Store custom emojis as uploads. * Depending on a hardcoded directory was a flawed design which made it impossible to debug when custom emojis go missing.
2017-02-02 04:41:57 -05:00
CustomEmoji.create!(name: "trout", upload: Fabricate(:upload))
Fix failing specs.
2017-03-14 02:58:22 -04:00
Emoji.clear_cache
FEATURE: Add option for `ExcerptParser` to keep onebox source.
2017-04-10 04:11:58 -04:00
REFACTOR: Migrate markdown functionality in ES6
2016-06-14 14:31:51 -04:00
expect(PrettyText.cook("hello :trout:")).to match(/<img src[^>]+trout[^>]+>/)
end
FIX: Post does not save if it contains a tag link
2016-05-01 21:36:09 -04:00
end
FIX: allows to have custom emoji translation without static file (#9893)
2020-05-27 14:11:52 -04:00
describe "custom emoji translation" do
before do
PrettyText.reset_translations
SiteSetting.enable_emoji = true
SiteSetting.enable_emoji_shortcuts = true
plugin = Plugin::Instance.new
plugin.translate_emoji "0:)", "otter"
end
after do
Plugin::CustomEmoji.clear_cache
PrettyText.reset_translations
end
it "sets the custom translation" do
expect(PrettyText.cook("hello 0:)")).to match(/otter/)
end
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "replaces skin toned emoji" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
expect(PrettyText.cook("hello 👱🏿‍♀️")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/blonde_woman/6.png?v=#{Emoji::EMOJI_VERSION}\" title=\":blonde_woman:t6:\" class=\"emoji\" alt=\":blonde_woman:t6:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
expect(PrettyText.cook("hello 👩‍🎤")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/woman_singer.png?v=#{Emoji::EMOJI_VERSION}\" title=\":woman_singer:\" class=\"emoji\" alt=\":woman_singer:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
expect(PrettyText.cook("hello 👩🏾‍🎓")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/woman_student/5.png?v=#{Emoji::EMOJI_VERSION}\" title=\":woman_student:t5:\" class=\"emoji\" alt=\":woman_student:t5:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
expect(PrettyText.cook("hello 🤷‍♀️")).to eq(
"<p>hello <img src=\"/images/emoji/twitter/woman_shrugging.png?v=#{Emoji::EMOJI_VERSION}\" title=\":woman_shrugging:\" class=\"emoji\" alt=\":woman_shrugging:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>",
)
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
correct more edge cases with new engine
2017-07-17 17:25:40 -04:00
it "should not treat a non emoji as an emoji" do
expect(PrettyText.cook(":email,class_name:")).not_to include("emoji")
end
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports href schemes" do
SiteSetting.allowed_href_schemes = "macappstore|steam"
cooked = cook("[Steam URL Scheme](steam://store/452530)")
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expected =
'<p><a href="steam://store/452530" rel="noopener nofollow ugc">Steam URL Scheme</a></p>'
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(n expected)
end
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports forbidden schemes" do
SiteSetting.allowed_href_schemes = "macappstore|itunes"
cooked = cook("[Steam URL Scheme](steam://store/452530)")
expected = "<p><a>Steam URL Scheme</a></p>"
expect(cooked).to eq(n expected)
end
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: Apply 'allowed_href_schemes' to all src/srcset attributes (#16860) Previously we were only applying the restriction to `a[href]` and `img[src]`. This commit ensures we apply the same logic to all allowlisted media src attributes.
2022-05-19 06:18:30 -04:00
it "applies scheme restrictions to img[src] attributes" do
SiteSetting.allowed_href_schemes = "steam"
cooked =
cook "![Steam URL Image](steam://store/452530) ![Other scheme image](itunes://store/452530)"
expected =
'<p><img src="steam://store/452530" alt="Steam URL Image"> <img src="" alt="Other scheme image"></p>'
expect(cooked).to eq(n expected)
end
it "applies scheme restrictions to track[src] and source[src]" do
SiteSetting.allowed_href_schemes = "steam"
cooked = cook <<~MD
<video>
<source src="steam://store/452530"><source src="itunes://store/452530"><track src="steam://store/452530"><track src="itunes://store/452530">
</video>
MD
expect(cooked).to include <<~HTML
<source src="steam://store/452530"><source src=""><track src="steam://store/452530"><track src="">
HTML
end
it "applies scheme restrictions to source[srcset]" do
SiteSetting.allowed_href_schemes = "steam"
cooked = cook <<~MD
<video>
<source srcset="steam://store/452530 1x,itunes://store/123 2x"><source srcset="steam://store/452530"><source srcset="itunes://store/452530">
</video>
MD
expect(cooked).to include <<~HTML
<source srcset="steam://store/452530 1x,"><source srcset="steam://store/452530"><source srcset="">
HTML
end
FIX: allowed href scheme link can start with a + (#5537) * allowed href scheme link can start with a + * allow tel:// links only to start with + * add missing semicolon * add test
2018-01-29 19:02:23 -05:00
it "allows only tel URL scheme to start with a plus character" do
SiteSetting.allowed_href_schemes = "tel|steam"
cooked = cook("[Tel URL Scheme](tel://+452530579785)")
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
expected = '<p><a href="tel://+452530579785" rel="noopener nofollow ugc">Tel URL Scheme</a></p>'
FIX: allowed href scheme link can start with a + (#5537) * allowed href scheme link can start with a + * allow tel:// links only to start with + * add missing semicolon * add test
2018-01-29 19:02:23 -05:00
expect(cooked).to eq(n expected)
cooked2 = cook("[Steam URL Scheme](steam://+store/452530)")
expected2 = "<p><a>Steam URL Scheme</a></p>"
expect(cooked2).to eq(n expected2)
end
UX: Only include tag hashtag postfix when necessary. https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 01:54:50 -04:00
it "produces hashtag links" do
FIX: Fix incorrect hashtag setting migration (#19857) Added in c2013865d78d353280ae68135b4c218e21ffb566, this migration was supposed to only turn off the hashtag setting for existing sites (since that was the old default) but its doing it for new ones too because we run all migrations on new sites. Instead, we should only run this if the first migration was only just created, meaning its a new site.
2023-01-15 19:53:00 -05:00
# TODO (martin) Remove when enable_experimental_hashtag_autocomplete is default for all sites
SiteSetting.enable_experimental_hashtag_autocomplete = false
UX: Only include tag hashtag postfix when necessary. https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 01:54:50 -04:00
category = Fabricate(:category, name: "testing")
category2 = Fabricate(:category, name: "known")
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
Fabricate(:topic, tags: [Fabricate(:tag, name: "known")])
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
UX: Only include tag hashtag postfix when necessary. https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 01:54:50 -04:00
cooked = PrettyText.cook(" #unknown::tag #known #known::tag #testing")
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
UX: Only include tag hashtag postfix when necessary. https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 01:54:50 -04:00
[
"<span class=\"hashtag\">#unknown::tag</span>",
DEV: Deprecate Category#url_with_id in favor of Category#url (#9972)
2020-06-18 04:32:14 -04:00
"<a class=\"hashtag\" href=\"#{category2.url}\">#<span>known</span></a>",
FIX: Improve anchor links (#12683) * FIX: Use theme color for anchor icon * FIX: Do not count anchor links * FIX: Do not count hashtags links either * DEV: Add tests for link_count * FIX: Disable anchors in quotes and preview * FIX: Try building some anchor slugs for unicode * DEV: Fix tests
2021-04-14 03:27:07 -04:00
"<a class=\"hashtag\" href=\"/tag/known\">#<span>known</span></a>",
DEV: Deprecate Category#url_with_id in favor of Category#url (#9972)
2020-06-18 04:32:14 -04:00
"<a class=\"hashtag\" href=\"#{category.url}\">#<span>testing</span></a>",
UX: Only include tag hashtag postfix when necessary. https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 01:54:50 -04:00
].each { |element| expect(cooked).to include(element) }
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: Reference example.com instead of somesite.com in examples * somesite.com actually exists... * example.com should be used in examples and is harmless to visit
2018-06-19 10:25:10 -04:00
cooked = PrettyText.cook("[`a` #known::tag here](http://example.com)")
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
html = <<~HTML
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
<p><a href="http://example.com" rel="noopener nofollow ugc"><code>a</code> #known::tag here</a></p>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
HTML
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html.strip)
FIX: don't create tags autolinks inside links (new engine)
2017-07-07 13:03:36 -04:00
FIX: Reference example.com instead of somesite.com in examples * somesite.com actually exists... * example.com should be used in examples and is harmless to visit
2018-06-19 10:25:10 -04:00
cooked = PrettyText.cook("<a href='http://example.com'>`a` #known::tag here</a>")
FIX: don't create tags autolinks inside links (new engine)
2017-07-07 13:03:36 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html.strip)
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
cooked = PrettyText.cook("<A href='/a'>test</A> #known::tag")
html = <<~HTML
FIX: Improve anchor links (#12683) * FIX: Use theme color for anchor icon * FIX: Do not count anchor links * FIX: Do not count hashtags links either * DEV: Add tests for link_count * FIX: Disable anchors in quotes and preview * FIX: Try building some anchor slugs for unicode * DEV: Fix tests
2021-04-14 03:27:07 -04:00
<p><a href="/a">test</a> <a class="hashtag" href="/tag/known">#<span>known</span></a></p>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
HTML
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html.strip)
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
# ensure it does not fight with the autolinker
expect(PrettyText.cook(" http://somewhere.com/#known")).not_to include("hashtag")
expect(PrettyText.cook(" http://somewhere.com/?#known")).not_to include("hashtag")
expect(PrettyText.cook(" http://somewhere.com/?abc#known")).not_to include("hashtag")
end
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) This commit fleshes out and adds functionality for the new `#hashtag` search and lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete` feature flag. **Serverside** We have two plugin API registration methods that are used to define data sources (`register_hashtag_data_source`) and hashtag result type priorities depending on the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb should make it clear what these are doing. Reading the `HashtagAutocompleteService` in full will likely help a lot as well. Each data source is responsible for providing its own **lookup** and **search** method that returns hashtag results based on the arguments provided. For example, the category hashtag data source has to take into account parent categories and how they relate, and each data source has to define their own icon to use for the hashtag, and so on. The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`. There is `hashtag_icons` that is just a simple array of all the different icons that can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations` that is used to store the type priority orders for each registered context. When sending emails, we cannot render the SVG icons for hashtags, so we need to change the HTML hashtags to the normal `#hashtag` text. **Markdown** The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete` markdown rule, and like all of our rules this is used to cook the raw text on both the clientside and on the serverside using MiniRacer. Only on the server side do we actually reach out to the database with the `hashtagLookup` function, on the clientside we just render a plainer version of the hashtag HTML. Only in the composer preview do we do further lookups based on this. This rule is the first one (that I can find) that uses the `currentUser` based on a passed in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id` for both the post and chat message. In some cases we need to cook without a user present, so the `Discourse.system_user` is used in this case. **Chat Channels** This also contains the changes required for chat so that chat channels can be used as a data source for hashtag searches and lookups. This data source will only be used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have to worry about channel results suddenly turning up. ------ **Known Rough Edges** - Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR - Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR - Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future - Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity) - Additional refinements and review fixes wil
2022-11-20 17:37:06 -05:00
it "produces hashtag links when enable_experimental_hashtag_autocomplete is enabled" do
SiteSetting.enable_experimental_hashtag_autocomplete = true
user = Fabricate(:user)
FIX: Server-side hashtag lookups of secure categories for a user (#19377) * FIX: Use Category.secured(guardian) for hashtag datasource Follow up to comments in #19219, changing the category hashtag datasource to use Category.secured(guardian) instead of Site.new(guardian).categories here since the latter does more work for not much benefit, and the query time is the same. Also eliminates some Hash -> Model back and forth busywork. Add some more specs too. * FIX: Server-side hashtag lookup cooking user loading When we were using the PrettyText.options.currentUser and parsing back and forth with JSON for the hashtag lookups server-side, we had a bug where the user's secure categories were not loaded since we never actually loaded a User model from the database, only parsed it from JSON. This commit fixes the issue by instead using the PretyText.options.userId and looking up the user directly from the database when calling hashtag_lookup via the PrettyText::Helpers code when cooking server-side. Added the missing spec to check for this as well.
2022-12-08 19:34:25 -05:00
category = Fabricate(:category, name: "testing", slug: "testing")
category2 = Fabricate(:category, name: "known", slug: "known")
group = Fabricate(:group)
private_category = Fabricate(:private_category, name: "secret", group: group, slug: "secret")
FIX: Do not cook icon with hashtags (#21676) This commit makes some fundamental changes to how hashtag cooking and icon generation works in the new experimental hashtag autocomplete mode. Previously we cooked the appropriate SVG icon with the cooked hashtag, though this has proved inflexible especially for theming purposes. Instead, we now cook a data-ID attribute with the hashtag and add a new span as an icon placeholder. This is replaced on the client side with an icon (or a square span in the case of categories) on the client side via the decorateCooked API for posts and chat messages. This client side logic uses the generated hashtag, category, and channel CSS classes added in a previous commit. This is missing changes to the sidebar to use the new generated CSS classes and also colors and the split square for categories in the hashtag autocomplete menu -- I will tackle this in a separate PR so it is clearer.
2023-05-23 03:33:55 -04:00
tag = Fabricate(:tag, name: "known")
Fabricate(:topic, tags: [tag])
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) This commit fleshes out and adds functionality for the new `#hashtag` search and lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete` feature flag. **Serverside** We have two plugin API registration methods that are used to define data sources (`register_hashtag_data_source`) and hashtag result type priorities depending on the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb should make it clear what these are doing. Reading the `HashtagAutocompleteService` in full will likely help a lot as well. Each data source is responsible for providing its own **lookup** and **search** method that returns hashtag results based on the arguments provided. For example, the category hashtag data source has to take into account parent categories and how they relate, and each data source has to define their own icon to use for the hashtag, and so on. The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`. There is `hashtag_icons` that is just a simple array of all the different icons that can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations` that is used to store the type priority orders for each registered context. When sending emails, we cannot render the SVG icons for hashtags, so we need to change the HTML hashtags to the normal `#hashtag` text. **Markdown** The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete` markdown rule, and like all of our rules this is used to cook the raw text on both the clientside and on the serverside using MiniRacer. Only on the server side do we actually reach out to the database with the `hashtagLookup` function, on the clientside we just render a plainer version of the hashtag HTML. Only in the composer preview do we do further lookups based on this. This rule is the first one (that I can find) that uses the `currentUser` based on a passed in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id` for both the post and chat message. In some cases we need to cook without a user present, so the `Discourse.system_user` is used in this case. **Chat Channels** This also contains the changes required for chat so that chat channels can be used as a data source for hashtag searches and lookups. This data source will only be used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have to worry about channel results suddenly turning up. ------ **Known Rough Edges** - Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR - Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR - Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future - Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity) - Additional refinements and review fixes wil
2022-11-20 17:37:06 -05:00
FIX: Server-side hashtag lookups of secure categories for a user (#19377) * FIX: Use Category.secured(guardian) for hashtag datasource Follow up to comments in #19219, changing the category hashtag datasource to use Category.secured(guardian) instead of Site.new(guardian).categories here since the latter does more work for not much benefit, and the query time is the same. Also eliminates some Hash -> Model back and forth busywork. Add some more specs too. * FIX: Server-side hashtag lookup cooking user loading When we were using the PrettyText.options.currentUser and parsing back and forth with JSON for the hashtag lookups server-side, we had a bug where the user's secure categories were not loaded since we never actually loaded a User model from the database, only parsed it from JSON. This commit fixes the issue by instead using the PretyText.options.userId and looking up the user directly from the database when calling hashtag_lookup via the PrettyText::Helpers code when cooking server-side. Added the missing spec to check for this as well.
2022-12-08 19:34:25 -05:00
cooked = PrettyText.cook(" #unknown::tag #known #known::tag #testing #secret", user_id: user.id)
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) This commit fleshes out and adds functionality for the new `#hashtag` search and lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete` feature flag. **Serverside** We have two plugin API registration methods that are used to define data sources (`register_hashtag_data_source`) and hashtag result type priorities depending on the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb should make it clear what these are doing. Reading the `HashtagAutocompleteService` in full will likely help a lot as well. Each data source is responsible for providing its own **lookup** and **search** method that returns hashtag results based on the arguments provided. For example, the category hashtag data source has to take into account parent categories and how they relate, and each data source has to define their own icon to use for the hashtag, and so on. The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`. There is `hashtag_icons` that is just a simple array of all the different icons that can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations` that is used to store the type priority orders for each registered context. When sending emails, we cannot render the SVG icons for hashtags, so we need to change the HTML hashtags to the normal `#hashtag` text. **Markdown** The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete` markdown rule, and like all of our rules this is used to cook the raw text on both the clientside and on the serverside using MiniRacer. Only on the server side do we actually reach out to the database with the `hashtagLookup` function, on the clientside we just render a plainer version of the hashtag HTML. Only in the composer preview do we do further lookups based on this. This rule is the first one (that I can find) that uses the `currentUser` based on a passed in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id` for both the post and chat message. In some cases we need to cook without a user present, so the `Discourse.system_user` is used in this case. **Chat Channels** This also contains the changes required for chat so that chat channels can be used as a data source for hashtag searches and lookups. This data source will only be used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have to worry about channel results suddenly turning up. ------ **Known Rough Edges** - Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR - Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR - Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future - Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity) - Additional refinements and review fixes wil
2022-11-20 17:37:06 -05:00
FIX: Add aria-label attribute to cooked hashtags (#22182) This commit adds an aria-label attribute to cooked hashtags using the post/chat message decorateCooked functionality. I have just used the inner content of the hashtag (the tag/category/channel name) for the label -- we can reexamine at some point if we want something different like "Link to dev category" or something, but from what I can tell things like Twitter don't even have aria-labels for hashtags so the text would be read out directly. This commit also refactors any ruby specs checking the HTML of hashtags to use rspec-html-matchers which is far clearer than having to maintain the HTML structure in a HEREDOC for comparison, and gives better spec failures. c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866/23?u=martin
2023-06-20 01:47:17 -04:00
expect(cooked).to have_tag("span", text: "#unknown::tag", with: { class: "hashtag-raw" })
expect(cooked).to have_tag(
"a",
with: {
class: "hashtag-cooked",
href: category2.url,
"data-type": "category",
"data-slug": category2.slug,
"data-id": category2.id,
},
) do
with_tag("span", with: { class: "hashtag-icon-placeholder" })
end
expect(cooked).to have_tag(
"a",
with: {
class: "hashtag-cooked",
href: category.url,
"data-type": "category",
"data-slug": category.slug,
"data-id": category.id,
},
) do
with_tag("span", with: { class: "hashtag-icon-placeholder" })
end
expect(cooked).to have_tag(
"a",
with: {
class: "hashtag-cooked",
href: tag.url,
"data-type": "tag",
"data-slug": tag.name,
"data-id": tag.id,
},
) do
with_tag("span", with: { class: "hashtag-icon-placeholder" })
end
expect(cooked).to have_tag("span", text: "#secret", with: { class: "hashtag-raw" })
FIX: Server-side hashtag lookups of secure categories for a user (#19377) * FIX: Use Category.secured(guardian) for hashtag datasource Follow up to comments in #19219, changing the category hashtag datasource to use Category.secured(guardian) instead of Site.new(guardian).categories here since the latter does more work for not much benefit, and the query time is the same. Also eliminates some Hash -> Model back and forth busywork. Add some more specs too. * FIX: Server-side hashtag lookup cooking user loading When we were using the PrettyText.options.currentUser and parsing back and forth with JSON for the hashtag lookups server-side, we had a bug where the user's secure categories were not loaded since we never actually loaded a User model from the database, only parsed it from JSON. This commit fixes the issue by instead using the PretyText.options.userId and looking up the user directly from the database when calling hashtag_lookup via the PrettyText::Helpers code when cooking server-side. Added the missing spec to check for this as well.
2022-12-08 19:34:25 -05:00
# If the user hash access to the private category it should be cooked with the details + icon
group.add(user)
cooked = PrettyText.cook(" #unknown::tag #known #known::tag #testing #secret", user_id: user.id)
FIX: Add aria-label attribute to cooked hashtags (#22182) This commit adds an aria-label attribute to cooked hashtags using the post/chat message decorateCooked functionality. I have just used the inner content of the hashtag (the tag/category/channel name) for the label -- we can reexamine at some point if we want something different like "Link to dev category" or something, but from what I can tell things like Twitter don't even have aria-labels for hashtags so the text would be read out directly. This commit also refactors any ruby specs checking the HTML of hashtags to use rspec-html-matchers which is far clearer than having to maintain the HTML structure in a HEREDOC for comparison, and gives better spec failures. c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866/23?u=martin
2023-06-20 01:47:17 -04:00
expect(cooked).to have_tag(
"a",
with: {
class: "hashtag-cooked",
href: private_category.url,
"data-type": "category",
"data-slug": private_category.slug,
"data-id": private_category.id,
},
) do
with_tag("span", with: { class: "hashtag-icon-placeholder" })
end
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) This commit fleshes out and adds functionality for the new `#hashtag` search and lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete` feature flag. **Serverside** We have two plugin API registration methods that are used to define data sources (`register_hashtag_data_source`) and hashtag result type priorities depending on the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb should make it clear what these are doing. Reading the `HashtagAutocompleteService` in full will likely help a lot as well. Each data source is responsible for providing its own **lookup** and **search** method that returns hashtag results based on the arguments provided. For example, the category hashtag data source has to take into account parent categories and how they relate, and each data source has to define their own icon to use for the hashtag, and so on. The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`. There is `hashtag_icons` that is just a simple array of all the different icons that can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations` that is used to store the type priority orders for each registered context. When sending emails, we cannot render the SVG icons for hashtags, so we need to change the HTML hashtags to the normal `#hashtag` text. **Markdown** The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete` markdown rule, and like all of our rules this is used to cook the raw text on both the clientside and on the serverside using MiniRacer. Only on the server side do we actually reach out to the database with the `hashtagLookup` function, on the clientside we just render a plainer version of the hashtag HTML. Only in the composer preview do we do further lookups based on this. This rule is the first one (that I can find) that uses the `currentUser` based on a passed in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id` for both the post and chat message. In some cases we need to cook without a user present, so the `Discourse.system_user` is used in this case. **Chat Channels** This also contains the changes required for chat so that chat channels can be used as a data source for hashtag searches and lookups. This data source will only be used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have to worry about channel results suddenly turning up. ------ **Known Rough Edges** - Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR - Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR - Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future - Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity) - Additional refinements and review fixes wil
2022-11-20 17:37:06 -05:00
cooked = PrettyText.cook("[`a` #known::tag here](http://example.com)", user_id: user.id)
html = <<~HTML
<p><a href="http://example.com" rel="noopener nofollow ugc"><code>a</code> #known::tag here</a></p>
HTML
expect(cooked).to eq(html.strip)
cooked =
PrettyText.cook("<a href='http://example.com'>`a` #known::tag here</a>", user_id: user.id)
expect(cooked).to eq(html.strip)
cooked = PrettyText.cook("<A href='/a'>test</A> #known::tag", user_id: user.id)
FIX: Add aria-label attribute to cooked hashtags (#22182) This commit adds an aria-label attribute to cooked hashtags using the post/chat message decorateCooked functionality. I have just used the inner content of the hashtag (the tag/category/channel name) for the label -- we can reexamine at some point if we want something different like "Link to dev category" or something, but from what I can tell things like Twitter don't even have aria-labels for hashtags so the text would be read out directly. This commit also refactors any ruby specs checking the HTML of hashtags to use rspec-html-matchers which is far clearer than having to maintain the HTML structure in a HEREDOC for comparison, and gives better spec failures. c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866/23?u=martin
2023-06-20 01:47:17 -04:00
expect(cooked).to have_tag(
"a",
with: {
class: "hashtag-cooked",
href: tag.url,
"data-type": "tag",
"data-slug": tag.name,
"data-id": tag.id,
},
) do
with_tag("span", with: { class: "hashtag-icon-placeholder" })
end
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) This commit fleshes out and adds functionality for the new `#hashtag` search and lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete` feature flag. **Serverside** We have two plugin API registration methods that are used to define data sources (`register_hashtag_data_source`) and hashtag result type priorities depending on the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb should make it clear what these are doing. Reading the `HashtagAutocompleteService` in full will likely help a lot as well. Each data source is responsible for providing its own **lookup** and **search** method that returns hashtag results based on the arguments provided. For example, the category hashtag data source has to take into account parent categories and how they relate, and each data source has to define their own icon to use for the hashtag, and so on. The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`. There is `hashtag_icons` that is just a simple array of all the different icons that can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations` that is used to store the type priority orders for each registered context. When sending emails, we cannot render the SVG icons for hashtags, so we need to change the HTML hashtags to the normal `#hashtag` text. **Markdown** The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete` markdown rule, and like all of our rules this is used to cook the raw text on both the clientside and on the serverside using MiniRacer. Only on the server side do we actually reach out to the database with the `hashtagLookup` function, on the clientside we just render a plainer version of the hashtag HTML. Only in the composer preview do we do further lookups based on this. This rule is the first one (that I can find) that uses the `currentUser` based on a passed in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id` for both the post and chat message. In some cases we need to cook without a user present, so the `Discourse.system_user` is used in this case. **Chat Channels** This also contains the changes required for chat so that chat channels can be used as a data source for hashtag searches and lookups. This data source will only be used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have to worry about channel results suddenly turning up. ------ **Known Rough Edges** - Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR - Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR - Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future - Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity) - Additional refinements and review fixes wil
2022-11-20 17:37:06 -05:00
# ensure it does not fight with the autolinker
expect(PrettyText.cook(" http://somewhere.com/#known")).not_to include("hashtag")
expect(PrettyText.cook(" http://somewhere.com/?#known")).not_to include("hashtag")
expect(PrettyText.cook(" http://somewhere.com/?abc#known")).not_to include("hashtag")
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle mixed lists" do
# known bug in old md engine
cooked = PrettyText.cook("* a\n\n1. b")
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-04 23:46:57 -04:00
expect(cooked).to match_html("<ul>\n<li>a</li>\n</ul>\n<ol>\n<li>b</li>\n</ol>")
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle traditional vs non traditional newlines" do
SiteSetting.traditional_markdown_linebreaks = true
expect(PrettyText.cook("1\n2")).to match_html "<p>1 2</p>"
FIX: mention not working after a newline (new engine)
2017-07-07 11:06:50 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
SiteSetting.traditional_markdown_linebreaks = false
expect(PrettyText.cook("1\n2")).to match_html "<p>1<br>\n2</p>"
end
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle emoji by name" do
expected = <<HTML
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
<p><img src="/images/emoji/twitter/smile.png?v=#{Emoji::EMOJI_VERSION}\" title=":smile:" class="emoji only-emoji" alt=":smile:" loading="lazy" width="20" height="20"><img src="/images/emoji/twitter/sunny.png?v=#{Emoji::EMOJI_VERSION}" title=":sunny:" class="emoji only-emoji" alt=":sunny:" loading="lazy" width="20" height="20"></p>
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
HTML
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(PrettyText.cook(":smile::sunny:")).to eq(expected.strip)
end
FIX: mention not working after a newline (new engine)
2017-07-07 11:06:50 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "handles emoji boundaries correctly" do
cooked = PrettyText.cook("a,:man:t2:,b")
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
expected =
"<p>a,<img src=\"/images/emoji/twitter/man/2.png?v=#{Emoji::EMOJI_VERSION}\" title=\":man:t2:\" class=\"emoji\" alt=\":man:t2:\" loading=\"lazy\" width=\"20\" height=\"20\">,b</p>"
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to match(expected.strip)
end
FEATURE: site setting for Markdown typographer It ships anyway with markdown.it so we might as well expose it
2017-06-27 16:50:13 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle emoji by translation" do
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
expected =
"<p><img src=\"/images/emoji/twitter/wink.png?v=#{Emoji::EMOJI_VERSION}\" title=\":wink:\" class=\"emoji only-emoji\" alt=\":wink:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>"
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(PrettyText.cook(";)")).to eq(expected)
end
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle multiple emojis by translation" do
cooked = PrettyText.cook(":) ;) :)")
expect(cooked.split("img").length - 1).to eq(3)
end
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
DEV: Correct typos and spelling mistakes (#12812) Over the years we accrued many spelling mistakes in the code base. This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change - comments - test descriptions - other low risk areas
2021-05-20 21:43:47 -04:00
it "handles emoji boundaries correctly" do
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(PrettyText.cook(",:)")).to include("emoji")
expect(PrettyText.cook(":-)\n")).to include("emoji")
expect(PrettyText.cook("a :)")).to include("emoji")
expect(PrettyText.cook(":),")).not_to include("emoji")
expect(PrettyText.cook("abcde ^:;-P")).to include("emoji")
end
corrected a parsing edge cases for bbcode blocks
2017-06-27 10:06:55 -04:00
FIX: Various watched words improvements - Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback) - Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins - Censor tests are moved to ruby, to ensure everything works end-to-end - If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-07-31 13:33:49 -04:00
describe "censoring" do
DEV: Don't user before(:all)/after(:all) (#13389) Leaking state and non-obvious order (before :all runs *before* RailsHelper.test_setup) are not worth it. A replacement PR for #13370. Fixes some flaky specs, e.g. ``` bin/rspec './spec/components/freedom_patches/translate_accelerator_spec.rb[1:3]' './spec/jobs/clean_up_user_export_topics_spec.rb[1:1]' --tag ~type:multisite --seed 35994 ``` Also included: * DEV: No need for locale reset (we do it anyway in rails_helper in `test_setup`)
2021-06-15 11:25:06 -04:00
after { Discourse.redis.flushdb }
FIX: Various watched words improvements - Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback) - Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins - Censor tests are moved to ruby, to ensure everything works end-to-end - If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-07-31 13:33:49 -04:00
def expect_cooked_match(raw, expected_cooked)
expect(PrettyText.cook(raw)).to eq(expected_cooked)
end
context "with basic words" do
fab!(:watched_words) do
%w[shucks whiz whizzer a**le badword* shuck$ café $uper].each do |word|
Fabricate(:watched_word, action: WatchedWord.actions[:censor], word: word)
end
end
it "works correctly" do
expect_cooked_match("aw shucks, golly gee whiz.", "<p>aw ■■■■■■, golly gee ■■■■.</p>")
end
it "doesn't censor words unless they have boundaries." do
expect_cooked_match(
"you are a whizzard! I love cheesewhiz. Whiz.",
"<p>you are a whizzard! I love cheesewhiz. ■■■■.</p>",
)
end
it "censors words even if previous partial matches exist." do
expect_cooked_match(
"you are a whizzer! I love cheesewhiz. Whiz.",
"<p>you are a ■■■■■■■! I love cheesewhiz. ■■■■.</p>",
)
end
it "won't break links by censoring them." do
expect_cooked_match(
"The link still works. [whiz](http://www.whiz.com)",
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
'<p>The link still works. <a href="http://www.whiz.com" rel="noopener nofollow ugc">■■■■</a></p>',
)
FIX: Various watched words improvements - Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback) - Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins - Censor tests are moved to ruby, to ensure everything works end-to-end - If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-07-31 13:33:49 -04:00
end
it "escapes regexp characters" do
expect_cooked_match("I have a pen, I have an a**le", "<p>I have a pen, I have an ■■■■■</p>")
end
it "works for words ending in non-word characters" do
expect_cooked_match(
"Aw shuck$, I can't fix the problem with money",
"<p>Aw ■■■■■■, I can't fix the problem with money</p>",
)
end
it "works for words ending in accented characters" do
expect_cooked_match("Let's go to a café today", "<p>Let's go to a ■■■■ today</p>")
end
it "works for words starting with non-word characters" do
expect_cooked_match("Discourse is $uper amazing", "<p>Discourse is ■■■■■ amazing</p>")
end
it "handles * as wildcard" do
expect_cooked_match("No badword or apple here plz.", "<p>No ■■■■■■■ or ■■■■■ here plz.</p>")
end
end
context "with watched words as regular expressions" do
before { SiteSetting.watched_words_regular_expressions = true }
it "supports words as regular expressions" do
%w[xyz* plee+ase].each do |word|
Fabricate(:watched_word, action: WatchedWord.actions[:censor], word: word)
end
expect_cooked_match(
"Pleased to meet you, but pleeeease call me later, xyz123",
"<p>Pleased to meet you, but ■■■■■■■■■ call me later, ■■■123</p>",
)
end
it "supports custom boundaries" do
Fabricate(:watched_word, action: WatchedWord.actions[:censor], word: "\\btown\\b")
expect_cooked_match(
"Meet downtown in your town at the townhouse on Main St.",
"<p>Meet downtown in your ■■■■ at the townhouse on Main St.</p>",
)
end
fix intermittent failing tests, some watched word refactoring
2017-07-27 12:26:55 -04:00
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
corrected a parsing edge cases for bbcode blocks
2017-06-27 10:06:55 -04:00
FIX: Split link watched words from replace (#13196) It was not clear that replace watched words can be used to replace text with URLs. This introduces a new watched word type that makes it easier to understand.
2021-06-02 01:36:49 -04:00
describe "watched words - replace & link" do
DEV: Don't user before(:all)/after(:all) (#13389) Leaking state and non-obvious order (before :all runs *before* RailsHelper.test_setup) are not worth it. A replacement PR for #13370. Fixes some flaky specs, e.g. ``` bin/rspec './spec/components/freedom_patches/translate_accelerator_spec.rb[1:3]' './spec/jobs/clean_up_user_export_topics_spec.rb[1:1]' --tag ~type:multisite --seed 35994 ``` Also included: * DEV: No need for locale reset (we do it anyway in rails_helper in `test_setup`)
2021-06-15 11:25:06 -04:00
after { Discourse.redis.flushdb }
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
it "replaces words with other words" do
FIX: Make replace watched words work with wildcard (#13084) Watched words are always regular expressions, despite watched_words_ _regular_expressions being enabled or not. Internally, wildcard characters are replaced with a regular expression that matches any non whitespace character.
2021-05-18 05:09:47 -04:00
Fabricate(
:watched_word,
action: WatchedWord.actions[:replace],
word: "dolor sit*",
replacement: "something else",
)
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
expect(PrettyText.cook("Lorem ipsum dolor sit amet")).to match_html(<<~HTML)
<p>Lorem ipsum something else amet</p>
HTML
FIX: Make replace watched words work with wildcard (#13084) Watched words are always regular expressions, despite watched_words_ _regular_expressions being enabled or not. Internally, wildcard characters are replaced with a regular expression that matches any non whitespace character.
2021-05-18 05:09:47 -04:00
expect(PrettyText.cook("Lorem ipsum dolor sits amet")).to match_html(<<~HTML)
<p>Lorem ipsum something else amet</p>
HTML
expect(PrettyText.cook("Lorem ipsum dolor sittt amet")).to match_html(<<~HTML)
<p>Lorem ipsum something else amet</p>
HTML
FIX: Add word boundaries to replace and tag watched words (#13405) The generated regular expressions did not contain \b which matched every text that contained the word, even if it was only a substring of a word. For example, if "art" was a watched word a post containing word "artist" matched.
2021-06-18 11:54:06 -04:00
expect(PrettyText.cook("Lorem ipsum xdolor sit amet")).to match_html(<<~HTML)
<p>Lorem ipsum xdolor sit amet</p>
HTML
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
end
it "replaces words with links" do
FIX: Split link watched words from replace (#13196) It was not clear that replace watched words can be used to replace text with URLs. This introduces a new watched word type that makes it easier to understand.
2021-06-02 01:36:49 -04:00
Fabricate(
:watched_word,
action: WatchedWord.actions[:link],
word: "meta",
replacement: "https://meta.discourse.org",
)
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
expect(PrettyText.cook("Meta is a Discourse forum")).to match_html(<<~HTML)
<p>
<a href=\"https://meta.discourse.org\" rel=\"noopener nofollow ugc\">Meta</a>
is a Discourse forum
</p>
HTML
end
it "works with regex" do
Fabricate(
:watched_word,
action: WatchedWord.actions[:replace],
word: "f.o",
replacement: "test",
)
expect(PrettyText.cook("foo")).to match_html("<p>foo</p>")
expect(PrettyText.cook("f.o")).to match_html("<p>test</p>")
SiteSetting.watched_words_regular_expressions = true
expect(PrettyText.cook("foo")).to match_html("<p>test</p>")
expect(PrettyText.cook("f.o")).to match_html("<p>test</p>")
end
FIX: Do not replace in mentions and hashtags (#14260) Watched words of type 'replace' or 'link' replaced the text inside mentions or hashtags too, which broke these. These types of watched words must skip any match that has an @ or # before it.
2021-09-09 05:03:59 -04:00
it "does not replace hashtags and mentions" do
FIX: Fix incorrect hashtag setting migration (#19857) Added in c2013865d78d353280ae68135b4c218e21ffb566, this migration was supposed to only turn off the hashtag setting for existing sites (since that was the old default) but its doing it for new ones too because we run all migrations on new sites. Instead, we should only run this if the first migration was only just created, meaning its a new site.
2023-01-15 19:53:00 -05:00
# TODO (martin) Remove when enable_experimental_hashtag_autocomplete is default for all sites
SiteSetting.enable_experimental_hashtag_autocomplete = false
FIX: Do not replace in mentions and hashtags (#14260) Watched words of type 'replace' or 'link' replaced the text inside mentions or hashtags too, which broke these. These types of watched words must skip any match that has an @ or # before it.
2021-09-09 05:03:59 -04:00
Fabricate(:user, username: "test")
category = Fabricate(:category, slug: "test")
Fabricate(
:watched_word,
action: WatchedWord.actions[:replace],
word: "test",
replacement: "discourse",
)
expect(PrettyText.cook("@test #test test")).to match_html(<<~HTML)
<p>
<a class="mention" href="/u/test">@test</a>
<a class="hashtag" href="/c/test/#{category.id}">#<span>test</span></a>
discourse
</p>
HTML
end
FIX: Do not replace words in hashtags and mentions (#14760) Watched words were replaced inside mentions and hashtags when watched word regular expressions were enabled.
2021-10-29 10:53:09 -04:00
it "does not replace hashtags and mentions when watched words are regular expressions" do
FIX: Fix incorrect hashtag setting migration (#19857) Added in c2013865d78d353280ae68135b4c218e21ffb566, this migration was supposed to only turn off the hashtag setting for existing sites (since that was the old default) but its doing it for new ones too because we run all migrations on new sites. Instead, we should only run this if the first migration was only just created, meaning its a new site.
2023-01-15 19:53:00 -05:00
# TODO (martin) Remove when enable_experimental_hashtag_autocomplete is default for all sites
SiteSetting.enable_experimental_hashtag_autocomplete = false
FIX: Do not replace words in hashtags and mentions (#14760) Watched words were replaced inside mentions and hashtags when watched word regular expressions were enabled.
2021-10-29 10:53:09 -04:00
SiteSetting.watched_words_regular_expressions = true
Fabricate(:user, username: "test")
FIX: Do not replace hashtag-cooked text with WatchedWords (#19279) Adds the .hashtag-cooked as an exception for watched words to not auto-link the text of the hashtag.
2022-12-01 01:31:06 -05:00
category = Fabricate(:category, slug: "test", name: "test")
FIX: Do not replace words in hashtags and mentions (#14760) Watched words were replaced inside mentions and hashtags when watched word regular expressions were enabled.
2021-10-29 10:53:09 -04:00
Fabricate(
:watched_word,
action: WatchedWord.actions[:replace],
word: "es",
replacement: "discourse",
)
FIX: Add aria-label attribute to cooked hashtags (#22182) This commit adds an aria-label attribute to cooked hashtags using the post/chat message decorateCooked functionality. I have just used the inner content of the hashtag (the tag/category/channel name) for the label -- we can reexamine at some point if we want something different like "Link to dev category" or something, but from what I can tell things like Twitter don't even have aria-labels for hashtags so the text would be read out directly. This commit also refactors any ruby specs checking the HTML of hashtags to use rspec-html-matchers which is far clearer than having to maintain the HTML structure in a HEREDOC for comparison, and gives better spec failures. c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866/23?u=martin
2023-06-20 01:47:17 -04:00
cooked = PrettyText.cook("@test #test test")
expect(cooked).to have_tag("a", text: "@test", with: { class: "mention", href: "/u/test" })
expect(cooked).to have_tag(
"a",
text: "#test",
with: {
class: "hashtag",
href: "/c/test/#{category.id}",
},
)
expect(cooked).to include("tdiscourset")
FIX: Do not replace hashtag-cooked text with WatchedWords (#19279) Adds the .hashtag-cooked as an exception for watched words to not auto-link the text of the hashtag.
2022-12-01 01:31:06 -05:00
SiteSetting.enable_experimental_hashtag_autocomplete = true
FIX: Add aria-label attribute to cooked hashtags (#22182) This commit adds an aria-label attribute to cooked hashtags using the post/chat message decorateCooked functionality. I have just used the inner content of the hashtag (the tag/category/channel name) for the label -- we can reexamine at some point if we want something different like "Link to dev category" or something, but from what I can tell things like Twitter don't even have aria-labels for hashtags so the text would be read out directly. This commit also refactors any ruby specs checking the HTML of hashtags to use rspec-html-matchers which is far clearer than having to maintain the HTML structure in a HEREDOC for comparison, and gives better spec failures. c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866/23?u=martin
2023-06-20 01:47:17 -04:00
cooked = PrettyText.cook("@test #test test")
expect(cooked).to have_tag("a", text: "@test", with: { class: "mention", href: "/u/test" })
expect(cooked).to have_tag(
"a",
text: "test",
with: {
class: "hashtag-cooked",
href: "/c/test/#{category.id}",
"data-type": "category",
"data-slug": category.slug,
"data-id": category.id,
},
) do
with_tag("span", with: { class: "hashtag-icon-placeholder" })
end
expect(cooked).to include("tdiscourset")
FIX: Do not replace words in hashtags and mentions (#14760) Watched words were replaced inside mentions and hashtags when watched word regular expressions were enabled.
2021-10-29 10:53:09 -04:00
end
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
it "supports overlapping words" do
FIX: Split link watched words from replace (#13196) It was not clear that replace watched words can be used to replace text with URLs. This introduces a new watched word type that makes it easier to understand.
2021-06-02 01:36:49 -04:00
Fabricate(
:watched_word,
action: WatchedWord.actions[:link],
word: "meta",
replacement: "https://meta.discourse.org",
)
Fabricate(:watched_word, action: WatchedWord.actions[:replace], word: "iz", replacement: "is")
Fabricate(
:watched_word,
action: WatchedWord.actions[:link],
word: "discourse",
replacement: "https://discourse.org",
)
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
FIX: Split link watched words from replace (#13196) It was not clear that replace watched words can be used to replace text with URLs. This introduces a new watched word type that makes it easier to understand.
2021-06-02 01:36:49 -04:00
expect(PrettyText.cook("Meta iz a Discourse forum")).to match_html(<<~HTML)
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
<p>
FIX: Split link watched words from replace (#13196) It was not clear that replace watched words can be used to replace text with URLs. This introduces a new watched word type that makes it easier to understand.
2021-06-02 01:36:49 -04:00
<a href="https://meta.discourse.org" rel="noopener nofollow ugc">Meta</a>
is a
Add watched words of type "replace" (#12020) This commit includes other various improvements to watched words. auto_silence_first_post_regex site setting was removed because it overlapped with 'require approval' watched words.
2021-02-25 07:00:58 -05:00
<a href="https://discourse.org" rel="noopener nofollow ugc">Discourse</a>
forum
</p>
HTML
end
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports typographer" do
SiteSetting.enable_markdown_typographer = true
FEATURE: Replace markdown-it replacements rule. (#12417) We override the default replacements rule to no longer replace "(c)", "(p)", and "(p)". Additionally, we merged the custom arrows rule into the replacement function.
2021-03-18 09:55:41 -04:00
expect(PrettyText.cook("->")).to eq("<p> → </p>")
correct spec, require closing tag for bbcode blocks
2017-06-26 16:50:46 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
SiteSetting.enable_markdown_typographer = false
FEATURE: Replace markdown-it replacements rule. (#12417) We override the default replacements rule to no longer replace "(c)", "(p)", and "(p)". Additionally, we merged the custom arrows rule into the replacement function.
2021-03-18 09:55:41 -04:00
expect(PrettyText.cook("->")).to eq("<p>-&gt;</p>")
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
FIX: stop using inline quote for local onebox
2017-06-27 10:57:29 -04:00
FEATURE: Site setting for typographic quotation marks Adds locale defaults for German and French
2019-07-11 17:15:35 -04:00
it "uses quotation marks from site settings" do
SiteSetting.enable_markdown_typographer = true
expect(PrettyText.cook(%q|"Do you know," he said, "what 'Discourse' is?"|)).to eq(
"<p>“Do you know,” he said, “what ‘Discourse’ is?”</p>",
)
SiteSetting.markdown_typographer_quotation_marks = "„|“|‚|‘"
expect(PrettyText.cook(%q|"Weißt du", sagte er, "was 'Discourse' ist?"|)).to eq(
"<p>„Weißt du“, sagte er, „was ‚Discourse‘ ist?“</p>",
)
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "handles onebox correctly" do
expect(PrettyText.cook("http://a.com\nhttp://b.com").split("onebox").length).to eq(3)
expect(PrettyText.cook("http://a.com\n\nhttp://b.com").split("onebox").length).to eq(3)
expect(PrettyText.cook("a\nhttp://a.com")).to include("onebox")
expect(PrettyText.cook("> http://a.com")).not_to include("onebox")
expect(PrettyText.cook("a\nhttp://a.com a")).not_to include("onebox")
expect(PrettyText.cook("a\nhttp://a.com\na")).to include("onebox")
expect(PrettyText.cook("http://a.com")).to include("onebox")
expect(PrettyText.cook("http://a.com ")).to include("onebox")
expect(PrettyText.cook("http://a.com a")).not_to include("onebox")
expect(PrettyText.cook("- http://a.com")).not_to include("onebox")
expect(PrettyText.cook("<http://a.com>")).not_to include("onebox")
expect(PrettyText.cook(" http://a.com")).not_to include("onebox")
expect(PrettyText.cook("a\n http://a.com")).not_to include("onebox")
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 08:27:28 -04:00
expect(PrettyText.cook("sam@sam.com")).not_to include("onebox")
correct more edge cases with new engine
2017-07-17 17:25:40 -04:00
expect(PrettyText.cook("<img src='a'>\nhttp://a.com")).to include("onebox")
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
FIX: stop using inline quote for local onebox
2017-06-27 10:57:29 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can handle bbcode" do
expect(PrettyText.cook("a[b]b[/b]c")).to eq('<p>a<span class="bbcode-b">b</span>c</p>')
expect(PrettyText.cook("a[i]b[/i]c")).to eq('<p>a<span class="bbcode-i">b</span>c</p>')
end
FIX: stop using inline quote for local onebox
2017-06-27 10:57:29 -04:00
FIX: Empty inline BBCodes were broken (#18276) Upgrading to Markdown.it v13 broke empty inline BBCodes. This works around the problem by adding an empty token before a closing token if the previous token was a BBCode token. It also removes the unused `jump` attribute which was removed in Markdown.it v12.3
2022-09-19 19:50:22 -04:00
it "supports empty inline BBCode" do
expect(PrettyText.cook("a[b][/b]c")).to eq('<p>a<span class="bbcode-b"></span>c</p>')
end
FIX: inline [code] not handled properly The text a [code]test[/code] Would eat up the `test` text cause translation from inline to block for replace rule was not properly handled
2018-04-26 01:18:22 -04:00
it "can handle bbcode after a newline" do
# this is not 100% ideal cause we get an extra p here, but this is pretty rare
expect(PrettyText.cook("a\n[code]code[/code]")).to eq(
"<p>a</p>\n<pre><code class=\"lang-auto\">code</code></pre>",
)
# this is fine
expect(PrettyText.cook("a\na[code]code[/code]")).to eq("<p>a<br>\na<code>code</code></p>")
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "can onebox local topics" do
PERF: Speed up tests (#15214)
2021-12-07 13:45:58 -05:00
op = post
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
reply = Fabricate(:post, topic_id: op.topic_id)
Correct logic so hashtags are properly ignored in links
2017-06-29 12:48:04 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
url = Discourse.base_url + reply.url
quote = create_post(topic_id: op.topic.id, raw: "This is a sample reply with a quote\n\n#{url}")
quote.reload
use custom whitelister for tables
2017-06-28 16:08:20 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(quote.cooked).not_to include("[quote")
end
use custom whitelister for tables
2017-06-28 16:08:20 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports tables" do
markdown = <<~MD
| Tables | Are | Cool |
| ------------- |:-------------:| -----:|
| col 3 is | right-aligned | $1600 |
MD
expected = <<~HTML
FEATURE: add overflow-y auto to Markdown tables
2017-11-13 01:52:15 -05:00
<div class="md-table">
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
<table>
<thead>
<tr>
<th>Tables</th>
<th style="text-align:center">Are</th>
<th style="text-align:right">Cool</th>
</tr>
</thead>
<tbody>
<tr>
<td>col 3 is</td>
<td style="text-align:center">right-aligned</td>
<td style="text-align:right">$1600</td>
</tr>
</tbody>
</table>
FEATURE: add overflow-y auto to Markdown tables
2017-11-13 01:52:15 -05:00
</div>
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
HTML
expect(PrettyText.cook(markdown)).to eq(expected.strip)
end
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports img bbcode" do
cooked = PrettyText.cook "[img]http://www.image/test.png[/img]"
A11Y: Set role=presentation if alt attr is missing (#18546) This applies to all images posted that do not have a user description.
2022-10-12 07:07:37 -04:00
html = "<p><img src=\"http://www.image/test.png\" alt=\"\" role=\"presentation\"></p>"
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html)
end
Support for [email] [url] [img] bbcodes
2017-06-29 13:59:40 -04:00
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
it "supports img bbcode entities in attributes" do
actual = PrettyText.cook "[img]http://aaa.com/?a=1&b=<script>alert(1);</script>[/img]"
expected =
'<p><img src="http://aaa.com/?a=1&b=&lt;script&gt;alert(1);&lt;/script&gt;" alt="" role="presentation"></p>'
expect(expected).to be_same_dom(actual)
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
end
Support for [email] [url] [img] bbcodes
2017-06-29 13:59:40 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports email bbcode" do
cooked = PrettyText.cook "[email]sam@sam.com[/email]"
html = '<p><a href="mailto:sam@sam.com" data-bbcode="true">sam@sam.com</a></p>'
expect(cooked).to eq(html)
end
Support for [email] [url] [img] bbcodes
2017-06-29 13:59:40 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports url bbcode" do
cooked = PrettyText.cook "[url]http://sam.com[/url]"
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
html =
'<p><a href="http://sam.com" data-bbcode="true" rel="noopener nofollow ugc">http://sam.com</a></p>'
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 08:27:28 -04:00
expect(cooked).to eq(html)
end
it "supports nesting tags in url" do
cooked = PrettyText.cook("[url=http://sam.com][b]I am sam[/b][/url]")
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
html =
'<p><a href="http://sam.com" data-bbcode="true" rel="noopener nofollow ugc"><span class="bbcode-b">I am sam</span></a></p>'
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html)
end
Support for [email] [url] [img] bbcodes
2017-06-29 13:59:40 -04:00
correct bbcode parsing edge case
2017-07-20 13:02:31 -04:00
it "supports query params in bbcode url" do
cooked =
PrettyText.cook(
"[url=https://www.amazon.com/Camcorder-Hausbell-302S-Control-Infrared/dp/B01KLOA1PI/?tag=discourse]BBcode link[/url]",
)
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
html =
'<p><a href="https://www.amazon.com/Camcorder-Hausbell-302S-Control-Infrared/dp/B01KLOA1PI/?tag=discourse" data-bbcode="true" rel="noopener nofollow ugc">BBcode link</a></p>'
correct bbcode parsing edge case
2017-07-20 13:02:31 -04:00
expect(cooked).to eq(html)
end
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports inline code bbcode" do
cooked = PrettyText.cook "Testing [code]codified **stuff** and `more` stuff[/code]"
html = "<p>Testing <code>codified **stuff** and `more` stuff</code></p>"
expect(cooked).to eq(html)
end
Support for [email] [url] [img] bbcodes
2017-06-29 13:59:40 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports block code bbcode" do
cooked = PrettyText.cook "[code]\ncodified\n\n\n **stuff** and `more` stuff\n[/code]"
html = "<pre><code class=\"lang-auto\">codified\n\n\n **stuff** and `more` stuff</code></pre>"
expect(cooked).to eq(html)
end
support for bbcode [code] blocks
2017-06-29 16:04:10 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "support special handling for space in urls" do
cooked = PrettyText.cook "http://testing.com?a%20b"
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
html =
'<p><a href="http://testing.com?a%20b" class="onebox" target="_blank" rel="noopener nofollow ugc">http://testing.com?a%20b</a></p>'
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html)
end
support for bbcode [code] blocks
2017-06-29 16:04:10 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "supports onebox for decoded urls" do
cooked = PrettyText.cook "http://testing.com?a%50b"
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
html =
'<p><a href="http://testing.com?a%50b" class="onebox" target="_blank" rel="noopener nofollow ugc">http://testing.com?aPb</a></p>'
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(cooked).to eq(html)
end
correct case where %20 decoded as space in auto link correct case where onebox is not applied when url is decoded
2017-07-03 16:32:53 -04:00
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 08:27:28 -04:00
it "should sanitize the html" do
expect(PrettyText.cook("<test>alert(42)</test>")).to eq "<p>alert(42)</p>"
end
it "should not onebox magically linked urls" do
expect(PrettyText.cook("[url]site.com[/url]")).not_to include("onebox")
end
correct case where %20 decoded as space in auto link correct case where onebox is not applied when url is decoded
2017-07-03 16:32:53 -04:00
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
it "should sanitize the html" do
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 08:27:28 -04:00
expect(PrettyText.cook("<p class='hi'>hi</p>")).to eq "<p>hi</p>"
end
it "should strip SCRIPT" do
FIX: smarter newline handling for <img> tags on line alone Run all of pretty text spec on new engine
2017-07-10 12:20:50 -04:00
expect(PrettyText.cook("<script>alert(42)</script>")).to eq ""
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
expect(PrettyText.cook("<div><script>alert(42)</script></div>")).to eq "<div></div>"
end
it "strips script regardless of sanitize" do
expect(
PrettyText.cook("<div><script>alert(42)</script></div>", sanitize: false),
).to eq "<div></div>"
Feature: CommonMark support This adds the markdown.it engine to Discourse. https://github.com/markdown-it/markdown-it As the migration is going to take a while the new engine is default disabled. To enable it you must change the hidden site setting: enable_experimental_markdown_it. This commit is a squash of many other commits, it also includes some improvements to autospec (ability to run plugins), and a dev dependency on the og gem for html normalization.
2017-06-08 18:02:30 -04:00
end
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 08:27:28 -04:00
it "should allow sanitize bypass" do
expect(
PrettyText.cook("<test>alert(42)</test>", sanitize: false),
).to eq "<p><test>alert(42)</test></p>"
end
FEATURE: support image dimensions via Markdown image
2017-07-11 12:13:03 -04:00
# custom rule used to specify image dimensions via alt tags
describe "image dimensions" do
it "allows title plus dimensions" do
cooked = PrettyText.cook <<~MD
![title with | title|220x100](http://png.com/my.png)
![](http://png.com/my.png)
![|220x100](http://png.com/my.png)
![stuff](http://png.com/my.png)
DEV: Improve `PrettyText` spec to test for markdown image title attr.
2019-06-09 23:00:23 -04:00
![|220x100,50%](http://png.com/my.png "some title")
FEATURE: support image dimensions via Markdown image
2017-07-11 12:13:03 -04:00
MD
html = <<~HTML
<p><img src="http://png.com/my.png" alt="title with | title" width="220" height="100"><br>
A11Y: Set role=presentation if alt attr is missing (#18546) This applies to all images posted that do not have a user description.
2022-10-12 07:07:37 -04:00
<img src="http://png.com/my.png" alt="" role="presentation"><br>
<img src="http://png.com/my.png" alt="" width="220" height="100" role="presentation"><br>
FEATURE: support image dimensions via Markdown image
2017-07-11 12:13:03 -04:00
<img src="http://png.com/my.png" alt="stuff"><br>
A11Y: Set role=presentation if alt attr is missing (#18546) This applies to all images posted that do not have a user description.
2022-10-12 07:07:37 -04:00
<img src="http://png.com/my.png" alt="" title="some title" width="110" height="50" role="presentation"></p>
FEATURE: support image dimensions via Markdown image
2017-07-11 12:13:03 -04:00
HTML
expect(cooked).to eq(html.strip)
end
Allow whitespace around % scaler in markdown image syntax (#5000)
2017-07-26 10:54:20 -04:00
FEAT: Allow image resize by width or height `|150x` resizes to 150px wide + auto-height. `x150` resizes to 150px tall and auto-width. Resize value can be from 1 to 999 (incl. for percentages).
2019-02-07 15:04:09 -05:00
it "ignores whitespace and allows scaling by percent, width, height" do
Allow whitespace around % scaler in markdown image syntax (#5000)
2017-07-26 10:54:20 -04:00
cooked = PrettyText.cook <<~MD
![|220x100, 50%](http://png.com/my.png)
![|220x100 , 50%](http://png.com/my.png)
![|220x100 ,50%](http://png.com/my.png)
FEAT: Allow image resize by width or height `|150x` resizes to 150px wide + auto-height. `x150` resizes to 150px tall and auto-width. Resize value can be from 1 to 999 (incl. for percentages).
2019-02-07 15:04:09 -05:00
![|220x100,150x](http://png.com/my.png)
![|220x100, x50](http://png.com/my.png)
Allow whitespace around % scaler in markdown image syntax (#5000)
2017-07-26 10:54:20 -04:00
MD
html = <<~HTML
A11Y: Set role=presentation if alt attr is missing (#18546) This applies to all images posted that do not have a user description.
2022-10-12 07:07:37 -04:00
<p><img src="http://png.com/my.png" alt="" width="110" height="50" role="presentation"><br>
<img src="http://png.com/my.png" alt="" width="110" height="50" role="presentation"><br>
<img src="http://png.com/my.png" alt="" width="110" height="50" role="presentation"><br>
<img src="http://png.com/my.png" alt="" width="150" height="68" role="presentation"><br>
<img src="http://png.com/my.png" alt="" width="110" height="50" role="presentation"></p>
Allow whitespace around % scaler in markdown image syntax (#5000)
2017-07-26 10:54:20 -04:00
HTML
FEATURE: support inline bbcode blocks eg: [quote]test[/quote]
2017-07-27 17:55:04 -04:00
expect(cooked).to eq(html.strip)
Allow whitespace around % scaler in markdown image syntax (#5000)
2017-07-26 10:54:20 -04:00
end
FEATURE: support image dimensions via Markdown image
2017-07-11 12:13:03 -04:00
end
FEATURE: Inline (Mini) Oneboxing see: https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400?source_topic_id=66066
2017-07-19 15:08:54 -04:00
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
describe "upload decoding" do
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
it "can decode upload:// for default setup" do
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
set_cdn_url("https://cdn.com")
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
upload = Fabricate(:upload)
raw = <<~RAW
![upload](#{upload.short_url})
FEATURE: Add base62 sha1 to cooked data attribute * FEATURE: Add base62 sha1 to data attribute in `Post#cooked`. * FIX: Use `Upload#short_url` when quoting an image.
2019-06-10 21:15:45 -04:00
![upload](#{upload.short_url} "some title to test")
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
- ![upload](#{upload.short_url})
- test
- ![upload](#{upload.short_url})
FIX: Incorrect/missing extension in short_url fails to map to upload. `Hash#invert` causes us to lose keys if the hash contains similar values.
2018-09-05 09:46:43 -04:00
DEV: enable frozen string literal on all files This reduces chances of errors where consumers of strings mutate inputs and reduces memory usage of the app. Test suite passes now, but there may be some stuff left, so we will run a few sites on a branch prior to merging
2019-05-02 18:17:27 -04:00
![upload](#{upload.short_url.gsub(".png", "")})
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
FEATURE: Support upload:// urls in img tags (#16277) Previously, our `upload://` protocol urls were only supported in markdown image tags. This meant that our PullHotlinkedImages job was forced to convert `<img` tags to markdown. Depending on the exact syntax, this can actually cause the image to break. This commit adds support for `upload://` inside regular HTML `<img` tags. In a future commit, we'll be able to use this to make our PullHotlinkedImages job much more robust. Context at https://meta.discourse.org/t/152801
2022-03-28 11:46:47 -04:00
Inline img <img src="#{upload.short_url}">
<div>
Block img <img src="#{upload.short_url}">
</div>
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
[some attachment](#{upload.short_url})
[some attachment|attachment](#{upload.short_url})
[some attachment|random](#{upload.short_url})
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
RAW
fix the incorrect specs Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 03:19:11 -04:00
cdn_url = Discourse.store.cdn_url(upload.url)
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
cooked = <<~HTML
fix the incorrect specs Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 03:19:11 -04:00
<p><img src="#{cdn_url}" alt="upload" data-base62-sha1="#{upload.base62_sha1}"></p>
<p><img src="#{cdn_url}" alt="upload" title="some title to test" data-base62-sha1="#{upload.base62_sha1}"></p>
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
<ul>
<li>
fix the incorrect specs Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 03:19:11 -04:00
<p><img src="#{cdn_url}" alt="upload" data-base62-sha1="#{upload.base62_sha1}"></p>
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
</li>
<li>
<p>test</p>
<ul>
fix the incorrect specs Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 03:19:11 -04:00
<li><img src="#{cdn_url}" alt="upload" data-base62-sha1="#{upload.base62_sha1}"></li>
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
</ul>
</li>
</ul>
fix the incorrect specs Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 03:19:11 -04:00
<p><img src="#{cdn_url}" alt="upload" data-base62-sha1="#{upload.base62_sha1}"></p>
FEATURE: Support upload:// urls in img tags (#16277) Previously, our `upload://` protocol urls were only supported in markdown image tags. This meant that our PullHotlinkedImages job was forced to convert `<img` tags to markdown. Depending on the exact syntax, this can actually cause the image to break. This commit adds support for `upload://` inside regular HTML `<img` tags. In a future commit, we'll be able to use this to make our PullHotlinkedImages job much more robust. Context at https://meta.discourse.org/t/152801
2022-03-28 11:46:47 -04:00
<p>Inline img <img src="#{cdn_url}" data-base62-sha1="#{upload.base62_sha1}"></p>
<div>
Block img <img src="#{cdn_url}" data-base62-sha1="#{upload.base62_sha1}">
</div>
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
<p><a href="#{upload.short_path}">some attachment</a></p>
<p><a class="attachment" href="#{upload.short_path}">some attachment</a></p>
<p><a href="#{upload.short_path}">some attachment|random</a></p>
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
HTML
expect(PrettyText.cook(raw)).to eq(cooked.strip)
end
it "can place a blank image if we can not find the upload" do
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
raw = <<~MD
![upload](upload://abcABC.png)
[some attachment|attachment](upload://abcdefg.png)
MD
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
cooked = <<~HTML
FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2. Previous attempt was missing `post_uploads` records.
2019-05-28 21:00:25 -04:00
<p><img src="/images/transparent.png" alt="upload" data-orig-src="upload://abcABC.png"></p>
DEV: Extend plugin API for uploads (#8440) * DEV: Add API to alter uploads Markdown * DEV: Extract data attributes from image / download Markdown For example '[test|attachment|hello=world]' will generate an 'a' element with a data attribute: 'data-hello=world'. This commit also makes MarkdownIt to transform '|attachment' into 'class="attachment"'. This transformation used to be a part of the process which resolves short URLs (i.e. upload://). * DEV: Export imageNameFromFileName
2019-12-09 09:20:03 -05:00
<p><a class="attachment" href="/404" data-orig-href="upload://abcdefg.png">some attachment</a></p>
FEATURE: server side support for upload:// markdown This allows uploads to be specified using short sha1 hash instead of full URL Client side change is pending
2017-08-22 11:46:15 -04:00
HTML
expect(PrettyText.cook(raw)).to eq(cooked.strip)
end
end
FIX: use allowlist and blocklist terminology (#10209) This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-26 20:23:54 -04:00
it "can properly allowlist iframes" do
FEATURE: allowed_iframes site setting for allowing iframes This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:34 -04:00
SiteSetting.allowed_iframes = "https://bob.com/a|http://silly.com?EMBED="
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
raw = <<~HTML
FEATURE: allowed_iframes site setting for allowing iframes This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:34 -04:00
<iframe src='https://www.google.com/maps/Embed?testing'></iframe>
<iframe src='https://bob.com/a?testing'></iframe>
<iframe src='HTTP://SILLY.COM?EMBED=111'></iframe>
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
HTML
FEATURE: allowed_iframes site setting for allowing iframes This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:34 -04:00
# we require explicit HTTPS here
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
html = <<~HTML
FEATURE: allowed_iframes site setting for allowing iframes This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:34 -04:00
<iframe src="https://bob.com/a?testing"></iframe>
<iframe src="HTTP://SILLY.COM?EMBED=111"></iframe>
DEV: Correctly tag heredocs (#16061) This allows text editors to use correct syntax coloring for the heredoc sections. Heredoc tag names we use: languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 14:50:55 -05:00
HTML
FEATURE: allowed_iframes site setting for allowing iframes This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:34 -04:00
cooked = PrettyText.cook(raw).strip
expect(cooked).to eq(html.strip)
end
FIX: skip iframe URLs with relative paths in pretty text sanitizer. (#21714) This commit prevents unallowed URLs in iframe src by adding a relative path like `https://bob.com/abc/def/../ghi`. Currently, the iframe linking to the site uses the current_user, not the post's author, so users who have no access to a certain path are not able to view anything they shouldn't.
2023-05-24 06:44:18 -04:00
it "can skip relative paths in allowlist iframes" do
SiteSetting.allowed_iframes = "https://bob.com/abc/def"
raw = <<~HTML
<iframe src='https://bob.com/abc/def'></iframe>
<iframe src='https://bob.com/abc/def/../ghi'></iframe>
<iframe src='https://bob.com/abc/def/ghi/../../jkl'></iframe>
HTML
html = <<~HTML
<iframe src="https://bob.com/abc/def"></iframe>
HTML
expect(PrettyText.cook(raw).strip).to eq(html.strip)
end
FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify New site settings: enable_markdown_linkify: which is default on, auto links https:// and http:// and mail:// markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-01-31 21:22:38 -05:00
it "You can disable linkify" do
md = "www.cnn.com test.it http://test.com https://test.ab https://a"
cooked = PrettyText.cook(md)
html = <<~HTML
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
<p><a href="http://www.cnn.com" rel="noopener nofollow ugc">www.cnn.com</a> test.it <a href="http://test.com" rel="noopener nofollow ugc">http://test.com</a> <a href="https://test.ab" rel="noopener nofollow ugc">https://test.ab</a> <a href="https://a" rel="noopener nofollow ugc">https://a</a></p>
FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify New site settings: enable_markdown_linkify: which is default on, auto links https:// and http:// and mail:// markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-01-31 21:22:38 -05:00
HTML
expect(cooked).to eq(html.strip)
# notice how cnn.com is no longer linked but it is
SiteSetting.markdown_linkify_tlds = "not_com|it"
cooked = PrettyText.cook(md)
html = <<~HTML
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
<p>www.cnn.com <a href="http://test.it" rel="noopener nofollow ugc">test.it</a> <a href="http://test.com" rel="noopener nofollow ugc">http://test.com</a> <a href="https://test.ab" rel="noopener nofollow ugc">https://test.ab</a> <a href="https://a" rel="noopener nofollow ugc">https://a</a></p>
FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify New site settings: enable_markdown_linkify: which is default on, auto links https:// and http:// and mail:// markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-01-31 21:22:38 -05:00
HTML
expect(cooked).to eq(html.strip)
# no tlds anymore
SiteSetting.markdown_linkify_tlds = ""
cooked = PrettyText.cook(md)
html = <<~HTML
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 11:59:51 -04:00
<p>www.cnn.com test.it <a href="http://test.com" rel="noopener nofollow ugc">http://test.com</a> <a href="https://test.ab" rel="noopener nofollow ugc">https://test.ab</a> <a href="https://a" rel="noopener nofollow ugc">https://a</a></p>
FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify New site settings: enable_markdown_linkify: which is default on, auto links https:// and http:// and mail:// markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-01-31 21:22:38 -05:00
HTML
expect(cooked).to eq(html.strip)
# lastly ... what about no linkify
SiteSetting.enable_markdown_linkify = false
cooked = PrettyText.cook(md)
html = <<~HTML
<p>www.cnn.com test.it http://test.com https://test.ab https://a</p>
HTML
end
DEV: Correct typos and spelling mistakes (#12812) Over the years we accrued many spelling mistakes in the code base. This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change - comments - test descriptions - other low risk areas
2021-05-20 21:43:47 -04:00
it "has a proper data whitelist on div" do
FEATURE: whitelist data for themes
2018-03-01 22:51:50 -05:00
cooked = PrettyText.cook("<div data-theme-a='a'>test</div>")
expect(cooked).to include("data-theme-a")
end
FIX: use allowlist and blocklist terminology (#10209) This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-26 20:23:54 -04:00
it "allowlists lang attribute" do
FEATURE: whitelist lang attribute
2018-08-02 02:53:08 -04:00
cooked =
PrettyText.cook(
"<p lang='fr'>tester</p><div lang='fr'>tester</div><span lang='fr'>tester</span>",
)
expect(cooked).to eq(
"<p lang=\"fr\">tester</p><div lang=\"fr\">tester</div><span lang=\"fr\">tester</span>",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 06:18:21 -05:00
)
FEATURE: whitelist lang attribute
2018-08-02 02:53:08 -04:00
end
FIX: use allowlist and blocklist terminology (#10209) This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-26 20:23:54 -04:00
it "allowlists ruby tags" do
FEATURE: allow ruby tags in Markdown
2018-08-02 21:47:36 -04:00
# read all about ruby chars at: https://en.wikipedia.org/wiki/Ruby_character
# basically it is super hard to remember every single rare letter when there are
# so many, so ruby tags provide a hint.
#
html = (<<~MD).strip
<ruby lang="je">
<rb lang="je">X</rb>
漢 <rp>(</rp><rt lang="je"> ㄏㄢˋ </rt><rp>)</rp>
</ruby>
MD
cooked = PrettyText.cook html
expect(cooked).to eq(html)
end
FEATURE: generic theme component bbcode wrapper (#7400) Usage: ``` [wrap=name foo=bar] hello world [/wrap] ```
2019-04-24 04:37:34 -04:00
describe "d-wrap" do
it "wraps the [wrap] tag inline" do
cooked = PrettyText.cook("[wrap=toc]taco[/wrap]")
html = <<~HTML
<div class="d-wrap" data-wrap="toc">
<p>taco</p>
</div>
HTML
expect(cooked).to eq(html.strip)
cooked = PrettyText.cook("Hello [wrap=toc id=1]taco[/wrap] world")
html = <<~HTML
<p>Hello <span class="d-wrap" data-wrap="toc" data-id="1">taco</span> world</p>
HTML
expect(cooked).to eq(html.strip)
end
it "wraps the [wrap] tag in block" do
FIX: compensate for typographer typographer can change " to ” leading to breakages in parser At least codify this. Longer term we want to re-prioritize typographer so it always runs after bbcode parsing.
2020-09-20 19:44:37 -04:00
# can interfere with parsing
SiteSetting.enable_markdown_typographer = true
FEATURE: generic theme component bbcode wrapper (#7400) Usage: ``` [wrap=name foo=bar] hello world [/wrap] ```
2019-04-24 04:37:34 -04:00
md = <<~MD
FIX: compensate for typographer typographer can change " to ” leading to breakages in parser At least codify this. Longer term we want to re-prioritize typographer so it always runs after bbcode parsing.
2020-09-20 19:44:37 -04:00
[wrap=toc id="a” aa='b"' bb="f'"]
taco1
FEATURE: generic theme component bbcode wrapper (#7400) Usage: ``` [wrap=name foo=bar] hello world [/wrap] ```
2019-04-24 04:37:34 -04:00
[/wrap]
MD
cooked = PrettyText.cook(md)
html = <<~HTML
FIX: compensate for typographer typographer can change " to ” leading to breakages in parser At least codify this. Longer term we want to re-prioritize typographer so it always runs after bbcode parsing.
2020-09-20 19:44:37 -04:00
<div class="d-wrap" data-wrap="toc" data-id="a" data-aa="b&amp;quot;" data-bb="f'">
<p>taco1</p>
FEATURE: generic theme component bbcode wrapper (#7400) Usage: ``` [wrap=name foo=bar] hello world [/wrap] ```
2019-04-24 04:37:34 -04:00
</div>
HTML
expect(cooked).to eq(html.strip)
end
it "wraps the [wrap] tag without content" do
md = <<~MD
[wrap=toc]
[/wrap]
MD
cooked = PrettyText.cook(md)
html = <<~HTML
<div class="d-wrap" data-wrap="toc"></div>
HTML
expect(cooked).to eq(html.strip)
end
it "adds attributes as data-attributes" do
FIX: Nested quotes in BBCode Previously attributes such as `[test a='a"a' b="a'a"]` were not correctly handled. This amends the regex parser to ensure it correctly parses attributes without breaking incorrectly on the first nested quote
2020-09-20 18:55:56 -04:00
cooked = PrettyText.cook("[wrap=toc name=\"single quote's\" id='1\"2']taco[/wrap]")
FEATURE: generic theme component bbcode wrapper (#7400) Usage: ``` [wrap=name foo=bar] hello world [/wrap] ```
2019-04-24 04:37:34 -04:00
html = <<~HTML
FIX: Nested quotes in BBCode Previously attributes such as `[test a='a"a' b="a'a"]` were not correctly handled. This amends the regex parser to ensure it correctly parses attributes without breaking incorrectly on the first nested quote
2020-09-20 18:55:56 -04:00
<div class="d-wrap" data-wrap="toc" data-name="single quote's" data-id="1&amp;quot;2">
FEATURE: generic theme component bbcode wrapper (#7400) Usage: ``` [wrap=name foo=bar] hello world [/wrap] ```
2019-04-24 04:37:34 -04:00
<p>taco</p>
</div>
HTML
expect(cooked).to eq(html.strip)
end
it "prevents xss" do
cooked = PrettyText.cook('[wrap=toc foo="<script>console.log(1)</script>"]taco[/wrap]')
html = <<~HTML
<div class="d-wrap" data-wrap="toc" data-foo="&amp;lt;script&amp;gt;console.log(1)&amp;lt;/script&amp;gt;">
<p>taco</p>
</div>
HTML
expect(cooked).to eq(html.strip)
end
it "allows a limited set of attributes chars" do
cooked = PrettyText.cook('[wrap=toc fo@"èk-"!io=bar]taco[/wrap]')
html = <<~HTML
<div class=\"d-wrap\" data-wrap=\"toc\" data-io=\"bar\">
<p>taco</p>
</div>
HTML
expect(cooked).to eq(html.strip)
end
end
FEATURE: Add anchor links to headings (#12379)
2021-03-23 04:45:06 -04:00
it "adds anchor links to headings" do
cooked = PrettyText.cook("# Hello world")
html = <<~HTML
<h1>
FIX: Generate unique HTML heading names (#12705) Headings with the exact same name generated exactly the same heading names, which was invalid. This replaces the old code for generating names for non-English headings which were using URI encode and resulted in unreadable headings.
2021-04-16 03:54:19 -04:00
<a name="hello-world-1" class="anchor" href="#hello-world-1"></a>
FEATURE: Add anchor links to headings (#12379)
2021-03-23 04:45:06 -04:00
Hello world
</h1>
HTML
expect(cooked).to match_html(html)
end
FEATURE: Customizable rules and plugins for `PrettyText.markdown`. This commit extends the options which can be passed to `PrettyText.markdown` so that which Markdown-it rules and Discourse Markdown plugins to be used when rendering a text can be customizable. Currently, this extension is mainly used by plugins.
2022-01-06 02:27:12 -05:00
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
describe "customizing markdown-it rules" do
FEATURE: Customizable rules and plugins for `PrettyText.markdown`. This commit extends the options which can be passed to `PrettyText.markdown` so that which Markdown-it rules and Discourse Markdown plugins to be used when rendering a text can be customizable. Currently, this extension is mainly used by plugins.
2022-01-06 02:27:12 -05:00
it "customizes the markdown-it rules correctly" do
cooked = PrettyText.cook("This is some text **bold**", markdown_it_rules: [])
expect(cooked).to eq("<p>This is some text **bold**</p>")
cooked = PrettyText.cook("This is some text **bold**", markdown_it_rules: ["emphasis"])
expect(cooked).to eq("<p>This is some text <strong>bold</strong></p>")
end
end
DEV: Use proper wording for contexts in specs
2022-07-27 12:14:14 -04:00
describe "enabling/disabling features" do
DEV: Fix "overridden" typos (#16399) There are still some, but those are in actual code that's used outside core, so the change there would need to go through the deprecation cycle. That's a task for another day.
2022-04-06 17:17:20 -04:00
it "allows features to be overridden" do
FEATURE: Customizable rules and plugins for `PrettyText.markdown`. This commit extends the options which can be passed to `PrettyText.markdown` so that which Markdown-it rules and Discourse Markdown plugins to be used when rendering a text can be customizable. Currently, this extension is mainly used by plugins.
2022-01-06 02:27:12 -05:00
cooked = PrettyText.cook(":grin: @mention", features_override: [])
expect(cooked).to eq("<p>:grin: @mention</p>")
cooked = PrettyText.cook(":grin: @mention", features_override: ["emoji"])
DEV: add native lazy loading for emojis (#15830)
2022-02-09 06:18:59 -05:00
expect(cooked).to eq(
"<p><img src=\"/images/emoji/twitter/grin.png?v=#{Emoji::EMOJI_VERSION}\" title=\":grin:\" class=\"emoji\" alt=\":grin:\" loading=\"lazy\" width=\"20\" height=\"20\"> @mention</p>",
)
FEATURE: Customizable rules and plugins for `PrettyText.markdown`. This commit extends the options which can be passed to `PrettyText.markdown` so that which Markdown-it rules and Discourse Markdown plugins to be used when rendering a text can be customizable. Currently, this extension is mainly used by plugins.
2022-01-06 02:27:12 -05:00
cooked = PrettyText.cook(":grin: @mention", features_override: %w[mentions text-post-process])
expect(cooked).to eq("<p>:grin: <span class=\"mention\">@mention</span></p>")
end
end
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-19 21:49:22 -04:00
it "does not amend HTML when scrubbing" do
md = <<~MD
<s>\n\nhello\n\n</s>
MD
html = <<~HTML
<s>\n<p>hello</p>\n</s>
HTML
cooked = PrettyText.cook(md)
expect(cooked.strip).to eq(html.strip)
end
Initial release of Discourse
2013-02-05 14:16:51 -05:00
end