Commit Graph

248 Commits

Author SHA1 Message Date
Sam c7b0764089 correct more edge cases with new engine 2017-07-17 17:44:48 -04:00
Sam 8921058c67 FIX: medium URL with @ incorrectly handled as mention
Also:

- remove unused code
- rename bbcode_ruler to bbcode.ruler
- add md.core.textPostProcess.ruler to apply at end of chain (excluding links)
2017-07-17 17:44:48 -04:00
Sam Saffron d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam 79a084dd58 Revert "remove old markdown engine work-in-progress"
This reverts commit ee470b5317.
2017-07-12 18:10:51 -04:00
Sam bcbb9f208d Revert "Integrate new engine, correct old specs"
This reverts commit f1b38ba4fb.
2017-07-12 18:10:07 -04:00
Sam f1b38ba4fb Integrate new engine, correct old specs
corrects edge cases with

- full quotes
- [url] with nested tags
- engine overrides
- onebox applying to non http srcs
2017-07-12 17:44:40 -04:00
Sam Saffron ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
Sam 5d139e461c FIX: multi option poll not working (new engine) 2017-07-11 14:43:34 -04:00
Sam 3f950a756a FEATURE: support image dimensions via Markdown image 2017-07-11 12:13:03 -04:00
Sam ba9898c5a9 FIX: smarter newline handling for <img> tags on line alone
Run all of pretty text spec on new engine
2017-07-10 12:22:15 -04:00
Sam 6f09df0deb FIX: don't create tags autolinks inside links (new engine) 2017-07-07 13:04:25 -04:00
Sam 74f6894606 oops checked in a broken test 2017-07-07 11:08:04 -04:00
Sam 436b894f7a FIX: mention not working after a newline (new engine) 2017-07-07 11:06:50 -04:00
Guo Xiang Tan 13f3de4bf6 Nuke all `SiteSetting.stubs` from our codebase. 2017-07-07 15:09:14 +09:00
Sam d2ba543ed3 correct case where %20 decoded as space in auto link
correct case where onebox is not applied when url is decoded
2017-07-03 16:32:53 -04:00
Sam d941ed90d6 support for bbcode [code] blocks 2017-06-29 16:04:10 -04:00
Sam de50d8cd35 Support for [email] [url] [img] bbcodes 2017-06-29 14:00:09 -04:00
Sam c6b83f4714 Correct logic so hashtags are properly ignored in links 2017-06-29 12:48:20 -04:00
Sam 0e391a72a1 use custom whitelister for tables 2017-06-28 16:08:29 -04:00
Sam cfbda863e8 implement unicode emoji replacements 2017-06-28 13:47:30 -04:00
Sam 4c5109ff5b FEATURE: site setting for Markdown typographer
It ships anyway with markdown.it so we might as well expose it
2017-06-27 16:50:13 -04:00
Sam 0d62420cbe Space prior to onebox stops onebox resolution 2017-06-27 16:16:17 -04:00
Sam 232de90607 explicit autolink should not be oneboxed 2017-06-27 15:34:16 -04:00
Sam 5897ae945a FIX: stop using inline quote for local onebox 2017-06-27 10:57:38 -04:00
Sam 0e0092d738 corrected a parsing edge cases for bbcode blocks 2017-06-27 10:07:06 -04:00
Sam 21f7ac1f25 correct spec, require closing tag for bbcode blocks 2017-06-26 16:51:55 -04:00
Sam 8967d50dc2 Clean up sanitization code
- remove html table test, this is soon to be deprecated
- move sanitization tests into pretty text.rb
- fix up whitelister so it makes a copy of options
2017-06-26 15:21:47 -04:00
Neil Lalonde 69b2d64334 FIX: letter avatars in quotes on subfolders installs are broken 2017-06-23 17:12:13 -04:00
Sam 814c8804d4 correct text highlighting for no highlight 2017-06-23 14:35:46 -04:00
Sam b7386958ed don't leak state into other tests 2017-06-23 12:01:33 -04:00
Sam 234694b50f Feature: CommonMark support
This adds the markdown.it engine to Discourse.
https://github.com/markdown-it/markdown-it

As the migration is going to take a while the new engine is default
disabled. To enable it you must change the hidden site setting:
enable_experimental_markdown_it.

This commit is a squash of many other commits, it also includes some
improvements to autospec (ability to run plugins), and a dev dependency
on the og gem for html normalization.
2017-06-23 12:01:33 -04:00
Joffrey JAFFEUX f1750be2be Fixes woman_shrugging name conversion 2017-06-15 08:26:29 +02:00
Joffrey JAFFEUX 95e1843d33 Improves native emoji conversion 2017-06-14 17:42:52 +02:00
Joffrey JAFFEUX ab6086bce9 Transforms native skin toned emojis to valid names 2017-06-14 15:35:37 +02:00
Joffrey JAFFEUX 09d34b7642 emoji images are now v5 not v3
TODO: we should not hardcode this
2017-06-05 16:10:36 +02:00
Guo Xiang Tan 3861bd2793 FIX: Quotes should be ignored when parsing for onebox source. 2017-04-11 15:22:21 +08:00
Guo Xiang Tan 0a4c30bce3 FIX: Handle cases where `alt` and `title` tag is blank when parsing excerpt. 2017-04-11 14:18:27 +08:00
Guo Xiang Tan e49f3a408e FEATURE: Add option for `ExcerptParser` to keep onebox source. 2017-04-10 16:11:58 +08:00
Robin Ward 14410b71fb Convert server side paths to use `/u/` 2017-03-30 10:23:24 -04:00
Guo Xiang Tan 6312b8b7e4 Fix failing specs. 2017-03-14 14:58:22 +08:00
Guo Xiang Tan 1a7e954e09 FIX: Store custom emojis as uploads.
* Depending on a hardcoded directory was a flawed design
  which made it impossible to debug when custom emojis go
  missing.
2017-03-14 13:07:18 +08:00
Régis Hanol ba115480ba FIX: wasn't extracting links to quoted posts 2017-02-06 14:45:04 +01:00
Guo Xiang Tan 1758af9a1d FIX: Perform emoji unescape for topic titles in quotes. 2017-01-11 17:23:13 +08:00
Guo Xiang Tan f473a119ff Remove unnecessary stub. 2017-01-06 08:53:30 +08:00
Arpit Jalan 2d0c99636a do not add rel noreferrer 2016-11-20 18:19:14 +05:30
Arpit Jalan 7cb76f7333 FIX: add rel noopener and noreferrer in addition to nofollow 2016-11-20 17:07:27 +05:30
Neil Lalonde 86522a52b7 FEATURE: add censored_pattern setting to censor posts using regex 2016-11-08 16:39:26 -05:00
Sam 89daa43754 FEATURE: remap emojis back for push notifications and desktop alerts 2016-10-11 13:03:48 +11:00
Robin Ward 3c12dd6549 FIX: Consider lazyYT divs as links when extracting 2016-09-22 16:50:24 -04:00
Robin Ward a546395397 REFACTOR: Migrate markdown functionality in ES6 2016-07-11 12:57:05 -04:00
Sam b15f6bd211 FIX: s3 cdn urls not remapped correctly 2016-06-30 18:58:38 +10:00
James Kiesel 7a6bc3f1d7 Apply notification styles to mailing list email manually (#4283)
* Apply notification styles to mailing list email manually

* Fix failing spec
2016-06-21 20:42:30 +05:30
Sam e01dc54f2a UX: we should always simply use emoji codes as opposed to treating as image 2016-06-02 12:29:25 +10:00
Neil Lalonde f13470b96b Use db schema for tags instead of plugin store and custom fields 2016-05-26 14:29:48 -04:00
James Kiesel feffe23cc5 FEATURE: More granular mailing list mode (#4068)
* Rearrange frontend to account for mailing list mode

* Allow update of user preference for mailing list frequency

* Add mailing list frequency estimate

* Simplify frequency estimate; disable activity summary for mailing list mode

* Remove combined updates

* Add specs for enqueue mailing list mode job

* Write mailing list method for mailer

* Fix linting error

* Account for stale topics

* Add translations for default mailing list setting

* One query for mailing list topics

* Fix failing spec

* WIP

* Flesh out html template

* First pass at text-based mailing list summary

* Add user avatar

* Properly format posts for mailing list

* Move make_all_links_absolute into Email::Styles

* Apply first_seen_at to user

* Send mailing list email summary hourly based on first_seen_at

* Branch and test cleanup

* Use existing mailing list mode estimate

* Fix failing specs
2016-05-21 15:17:54 +02:00
Sam d25dc126f7 FIX: Post does not save if it contains a tag link 2016-05-02 11:36:09 +10:00
Régis Hanol 1a621f3393 FIX: don't extract links to anchors 2016-04-15 20:02:18 +02:00
Régis Hanol 7acdbc8448 FIX: don't extract links from elided parts 2016-03-16 22:35:08 +01:00
Robin Ward 396713718b FEATURE: Adds ~200 more emoji
This includes all new emoji from Unicode 7 and Unicode 8 with the
exception of skin tones which will require a new interface.
2016-03-04 15:10:14 -05:00
Robin Ward 0167f6bb57 FIX: Don't substitute emojis within code blocks 2016-03-02 14:32:00 -05:00
Robert Riemann 9c39647cd2 fix: support for hyphens in group name
The group mention @ORG-team triggers notifications for the group @ORG. This fix changes the RegExp, so that the group name is correctly extracted.

see: https://meta.discourse.org/t/group-mentions-that-begin-with-the-same-characters-may-be-incorrect/39892/12?u=rriemann
2016-02-23 23:55:31 +01:00
Guo Xiang Tan a362ad9407 FIX: Emoji in Discourse onebox is wrapped in square brackets. 2016-01-13 19:00:11 +08:00
Robin Ward 74780c1688 Add more server side unicode replacements for emoji 2015-12-30 14:46:52 -05:00
Robin Ward c064dc1322 FEATURE: Perform a server side replacement of unicode emoji 2015-12-29 16:28:27 -05:00
Jeff Atwood e03861da7e change all emoji image tests to use ?v=1 2015-12-22 14:36:21 -08:00
Sam Saffron f97a754b30 Revert "Try adding an ALT tag for avatars" 2015-12-15 19:57:54 +11:00
Sam Saffron b7f6df7d0c fix specs and tests 2015-12-15 18:18:20 +11:00
Andy Waite 3e50313fdc Prepare for separation of RSpec helper files
Since rspec-rails 3, the default installation creates two helper files:
* `spec_helper.rb`
* `rails_helper.rb`

`spec_helper.rb` is intended as a way of running specs that do not
require Rails, whereas `rails_helper.rb` loads Rails (as Discourse's
current `spec_helper.rb` does).

For more information:

https://www.relishapp.com/rspec/rspec-rails/docs/upgrade#default-helper-files

In this commit, I've simply replaced all instances of `spec_helper` with
`rails_helper`, and renamed the original `spec_helper.rb`.

This brings the Discourse project closer to the standard usage of RSpec
in a Rails app.

At present, every spec relies on loading Rails, but there are likely
many that don't need to. In a future pull request, I hope to introduce a
separate, minimal `spec_helper.rb` which can be used in tests which
don't rely on Rails.
2015-12-01 20:39:42 +00:00
Sam 150ad01111 improve UX remove username said if we have a title 2015-09-25 14:18:38 +10:00
Sam e5234b38b8 FEATURE: add title expansion for off topic quotes 2015-09-25 13:35:14 +10:00
Sam 568adc49c0 FIX: fenced code blocks not hoisted correctly
also fixes unhoisting logic
2015-07-31 17:53:20 +10:00
Guo Xiang Tan 7c1e16da54 FEATURE: Display emojis in user stream. 2015-07-23 23:50:01 +08:00
Sam 4c92f05d8d more tests 2015-07-20 17:06:00 +10:00
Sam ecf50a741c strip classes from table instead of ignoring table 2015-07-20 17:05:24 +10:00
Sam e93665b9f7 FEATURE: site setting to allow html tables (which may come from imports)
(allow_html_tables , disabled by default)
2015-07-20 16:56:46 +10:00
Robin Ward b52e5d1536 FIX: `default_avatars` wasn't being used for some server side templates 2015-06-26 13:38:09 -04:00
Sam Saffron b7a0a295c0 FIX: s3 cdn would break cooking if <img> tag had no src 2015-06-10 19:28:21 +10:00
Sam 93ab03966e FIX: no-follow not handled correctly for sub domains
if a.com was whitelisted aa.com would pass through
2015-05-27 14:31:01 +10:00
Sam 90eaad336d FEATURE: allow users to pick a CDN for s3 assets 2015-05-26 11:13:12 +10:00
Ben Hadley-Evans c3b461f58d Add blank alt attribute to avatars.
This was giving an ugly border to avatars in the user card as the full size version loaded in Firefox.
2015-04-14 14:39:40 +01:00
Sam a82530012a FEATURE: Allow selection of highlight js languages
PERF: stop loading highlight js on load

To get latest highlight js run bin/rake highlightjs:update
2015-03-13 16:18:59 +11:00
Régis Hanol 6a68e8c272 FIX: use CDN for user card/profile background and user avatars (for real this time) 2015-01-29 22:53:48 +01:00
Régis Hanol b4e5937850 FIX: 🐛 ensure emoji are case insensitive 2015-01-15 19:00:55 +01:00
Luciano Sousa 0fd98b56d8 few components with rspec3 syntax 2015-01-09 13:34:37 -03:00
Sam e23e008682 FIX: buggy unhoisting of escaped \* 2014-12-23 18:25:10 +11:00
Régis Hanol 6027073547 FIX: properly unescape HTML entities in excerpts 2014-12-10 12:52:51 +01:00
Régis Hanol a5616146eb FIX: remove meta data from lightbox in both excerpt (html & text) 2014-11-05 20:37:00 +01:00
Régis Hanol ada750b384 fixed some more deprecations. 20 to go 2014-10-29 16:06:50 +01:00
Sam 0d1b460f1e Pending spec for Nokogiri fix 2014-10-03 12:37:07 +10:00
Régis Hanol de76b512c1 fix most deprecations in the specs (still some left) 2014-09-25 17:44:48 +02:00
Jeff Atwood 5646ebf4c7 fix tests expecting "name said:" to "name:" 2014-09-18 20:39:27 -07:00
Robin Ward 19b4364d79 SECURITY: Stripping links could unescape html fragments 2014-09-17 12:08:00 -04:00
David McClure d567093756 FEATURE: Allow manual excerpt to be specified anywhere in the post and override max excerpt length 2014-09-03 22:06:53 -07:00
David McClure 0513d02e23 FEATURE: Allow manual excerpt to be defined past the beginning of the post
There is still a limitation that the span excerpt must begin before the post_excerpt_max_length.
2014-09-03 00:29:49 -07:00
Sam 84836944e8 FIX: crash on invalid uri component 2014-07-30 17:09:55 +10:00
Robin Ward b2f2e7b1d2 REFACTOR: Move Markdown tests to Javascript land 2014-07-25 16:08:00 -04:00
Sam 6e9f5f5584 SECURITY: fix XSS in excerpt parser 2014-07-25 12:16:00 +10:00
Sam de7e6a9545 Feature: allow mods to cut pinned topic excerpts 2014-07-17 21:32:37 +10:00
Sam c12a131fb4 SECURITY: sanitizer allowing invalid attributes 2014-07-17 16:11:09 +10:00
Sam 89fc989adb FEATURE: First Quote badge 2014-07-11 14:17:43 +10:00
Sam d54c28adc1 FIX: better whitelisting 2014-07-10 09:59:54 +10:00
Sam 9828a268b9 Fix: whitelist regex for bbcode too wide 2014-07-10 09:17:04 +10:00
Robin Ward fc1ce96dbb FIX: Change the approach to sanitization. Includes a more detailed API
for allowing classes and attributes for only certain tag names.
2014-07-03 16:55:36 -04:00
Robin Ward 9c48f8f154 FIX: Don't surround `<aside>` with `<p>` as that is malformed HTML. 2014-06-30 18:11:22 -04:00
Sam e2e36a6df3 FIX: bold and italic handling improved 2014-06-30 17:01:46 +10:00
Sam 24ddb6cfad FIX: Bold, italic should not expect a space boundary 2014-06-26 17:45:51 +10:00
Sam b8357aa90a BUGFIX: newline after bold was not producting a BR 2014-06-26 15:28:08 +10:00
Robin Ward 64355c989e FIX: Don't extract links from empty quotes 2014-05-20 17:20:52 -04:00
Louis Rose 1574485443 Perform the where(...).first to find_by(...) refactoring.
This refactoring was automated using the command: bundle exec "ruby refactorings/where_dot_first_to_find_by/app.rb"
2014-05-06 14:41:59 +01:00
Robin Ward 87682f7539 FIX: Don't include image meta data when embedded in an email 2014-04-17 12:32:51 -04:00
riking 9c4dd1cb35 Change comma-delim site settings to pipe-delim 2014-04-08 14:17:55 -07:00
Vikhyat Korrapati e781a758ad Preserve spoiler tags in post excerpts. 2014-02-20 15:01:44 +05:30
Robin Ward 963793549c FIX: Remove failing ruby tests. The same thing is tested in JS and
passes due to the ability to create a document fragment.
2014-01-20 15:28:46 -05:00
Neil Lalonde 4f6b208e8d Posts by trust level 3 users do not have nofollow on their external links. 2014-01-15 11:40:51 -05:00
Régis Hanol ad8755aa70 BUGFIX: inline spoiler for text, block spoiler for images 2014-01-15 00:53:06 +01:00
Régis Hanol 70161498b6 BUGFIX: spoiler tag on lightboxed images wasn't working 2014-01-12 19:38:46 +01:00
Neil Lalonde 4ec0543362 FIX: emails with embedded posts should always use absolute URLs 2013-11-28 15:57:21 -05:00
Régis Hanol 37fd7ab574 pull hotlinked images 2013-11-05 19:07:29 +01:00
Robin Ward d7182d0b14 FIX: Only wrap inline html tags in <p> 2013-10-18 15:21:05 -04:00
Robin Ward f27413219e Support for MDTest 2013-10-16 10:28:42 -04:00
Robin Ward 5281b7f80c Upgraded and refactored Sanitizing. Much less crap should get through now!
Conflicts:
	app/assets/javascripts/discourse/components/syntax_highlighting.js
2013-10-15 10:53:11 -04:00
Robin Ward af931f0444 Reverting the Sanitizer commit in case we have to do something urgent
before we deploy it early next week. It's in the branch `sanitizer` for
now.

This reverts commit 9e93d8ed52.
2013-10-11 16:44:26 -04:00
Robin Ward 9e93d8ed52 Upgraded and refactored Sanitizing. Much less crap should get through now!
Conflicts:
	app/assets/javascripts/discourse/components/syntax_highlighting.js
2013-10-11 16:25:40 -04:00
Robin Ward 37304b7eba FIX: Too many new lines in long quotes 2013-09-27 15:08:56 -04:00
Robin Ward 3cec95a2c3 Better API for parsing out blocks in the parser. 2013-08-29 11:47:44 -04:00
Robin Ward 2d45c56ba5 Replace Markdown Linebreak Regexp with node parser. 2013-08-26 15:21:23 -04:00
Robin Ward 7c07079ed9 Revert "FIX: Failing spec"
This reverts commit 2df2838f40.
2013-08-26 12:46:28 -04:00
Robin Ward 2df2838f40 FIX: Failing spec 2013-08-26 11:54:05 -04:00
Robin Ward 7f69a58439 Replace Markdown parser. 2013-08-21 10:10:57 -04:00
Régis Hanol c867b67a0b custom avatar support 2013-08-13 22:08:29 +02:00
Robin Ward 751b757c1d FIX: Some failing specs 2013-07-18 12:26:38 -04:00
Régis Hanol 3136638b4b FIX: CDN doesn't work with S3 2013-07-16 22:16:33 +02:00
Sam c49731a91b fix failing server side quote localisation change 2013-07-16 17:48:48 +10:00
Robin Ward 7504da13e3 FIX: Whitespec in specs changed. We should really figure out a better way of ensuring correctness here. 2013-07-03 19:24:49 -04:00
Robin Ward 6384518599 Allow images in the daily digest for top scoring posts 2013-06-05 18:55:27 -04:00
Robin Ward 0b97ea6345 Better HTML emails, smarter email digests, new email section in admin with digest preview 2013-06-05 17:47:25 -04:00
Robin Ward dfba2b6e0a FIX: Strip links from google indexed bios when the users are new. 2013-06-05 15:28:10 -04:00
Régis Hanol 01855b70b4 FIX: Have onebox ignore internal links 2013-06-05 20:53:07 +02:00
Robin Ward 4392a17b54 Strip out links when displaying a new user's bio 2013-06-04 12:06:25 -04:00
Régis Hanol 297680c28d FIX: pinned topic excerpt is not properly truncated 2013-05-10 12:28:17 +02:00
Sam b25a5a20bb option to strip links from excerpts 2013-04-30 13:25:55 +10:00
Michael Scott Shappe 2fb673dd72 Create and use new rspec matcher 'match_html'
This introduces match_html, which converts actual and expected HTML strings into "canonical" HTML using Nokogiri with NOBLANKS and COMPACT, and then does a simple equality comparison.

This eliminates whitespace differences introduced by library changes (e.g. the libxml2 2.9.0 change).

pretty_text_spec.rb has been changed to use match_html where appropriate. and all tests pass under libxml2 2.7.6, 2.8.0 or 2.9.0
2013-04-09 22:30:10 -05:00
Robin Ward 00f9e628e0 Don't apply markdown to content of [quote] -- it messes up spacing. 2013-04-09 17:32:50 -04:00
Gosha Arinich cafc75b238 remove trailing whitespaces ❤️ 2013-02-26 07:31:35 +03:00
Robin Ward dee9e9a51f Fix bug where links to posts weren't being tracked 2013-02-13 15:23:09 -05:00
Sam Saffron f68f59c24f setting to exclude rel nofollow from particular domains 2013-02-11 18:58:19 +11:00
Sam Saffron 543845c673 rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default. 2013-02-11 11:43:07 +11:00
Robin Ward 21b5628528 Initial release of Discourse 2013-02-05 14:16:51 -05:00