Commit Graph

109 Commits

Author SHA1 Message Date
Blake Erickson 2d0ad48dd1
Revert "DEV: Add crossOrigin to video tag (#20617)" (#20624)
This reverts commit f6063c684b.

Videos on sites with a cdn enabled aren't playing w/ a default cdn
config. They are showing a "CORS request did not succeed" error.
2023-03-09 16:20:35 -07:00
Blake Erickson f6063c684b
DEV: Add crossOrigin to video tag (#20617)
* DEV: Add crossOrigin to video tag

This is a follow-up commit to f144c64e13
which enables the ability to generate thumbnail images for video
uploads.

In order for the html5 canvas element to create an image or blob the
source video element needs to to have the crossOrigin attribute set to
"anonymous" because a cdn is likely being used in production
environments.

We are already doing something similar in
e292c45924/app/assets/javascripts/discourse/app/lib/update-tab-count.js (L63)
2023-03-09 13:19:19 -07:00
Penar Musaraj c052df412a
DEV: Add support for aliases in HighlightJS languages (#20380)
We were only supporting the main name of each HighlightJS language. So, by default, you could not use `js` or `jsx` to highlight Javascript, given they are aliases for `javascript`.

This PR adds a list of aliases as a constant to core (built via a rake task), and then checks against the `highlighted_languages` site settings plus the list of aliases when processing a code block.
2023-02-23 15:06:06 -05:00
Osama Sayegh f91631b625
DEV: Update javascript:update_constants rake task following template colocation (#20365)
As of ba3f62f576, handlebars templates are colocated with js files so the path to hbs templates referenced by this rake task is no longer valid. This commit fixes the path to hbs templates and updates a couple of files that are generated by the rake task.
2023-02-20 06:20:47 +03:00
Martin Brennan 641a1a6b44
FIX: Allow for nested chat transcripts (#19572)
The way our markdown raw_html hoisting worked, we only
supported one level of hoisting the HTML content. However
when nesting [chat] transcript BBCode we need to allow
for multiple levels of it. This commit changes opts.discourse.hoisted
to be more constant, and the GUID keys that have the hoisted
content are only deleted by unhoistForCooked rather than
the cook function itself, which prematurely deletes them
when they are needed further down the line.
2022-12-23 09:56:30 +10:00
Roman Rizzi 9c8043a4d2
FEATURE: Enforce mention limits for chat messages (#19034)
* FEATURE: Enforce mention limits for chat messages

The first part of these changes adds a new setting called `max_mentions_per_chat_message`, which skips notifications when the message contains too many mentions. It also respects the `max_users_notified_per_group_mention` setting
and skips notifications if expanding a group mention would exceed it.

We also include a new component to display JIT warning for these limits to the user while composing a message.

* Simplify ignoring/muting filter in chat_notifier

* Post-send warnings for unsent warnings

* Improve pluralization

* Address review feedback

* Fix test

* Address second feedback round

* Third round of feedback

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2022-12-06 14:54:04 -03:00
Martin Brennan d3f02a1270
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.

**Serverside**

We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.

Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.

The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.

When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.

**Markdown**

The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.

This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.

**Chat Channels**

This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.

------

**Known Rough Edges**

- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 08:37:06 +10:00
David Taylor 392bafcd7e
DEV: Add unique ids to deprecations (#19019)
This will improve the QUnit deprecation summaries introduced via 8c48285145
2022-11-16 10:00:39 +00:00
Martin Brennan ac7bf98ad1
DEV: Load client site settings YML into JS tests (#18413)
Our method of loading a subset of client settings into tests via
tests/helpers/site-settings.js can be improved upon. Currently we have a
hardcoded subset of the client settings, which may get out of date and not have
the correct defaults. As well as this plugins do not get their settings into the
tests, so whenever you need a setting from a plugin, even if it has a default,
you have to do needs.setting({ ... }) which is inconvenient.

This commit introduces an ember CLI build step to take the site_settings.yml and
all the plugin settings.yml files, pull out the client settings, and dump them
into a variable in a single JS file we can load in our tests, so we have the
correct selection of settings and default values in our JS tests. It also fixes
many, many tests that were operating under incorrect assumptions or old
settings.

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2022-11-08 09:17:43 +10:00
Bianca Nenciu 33087f0bdf
A11Y: Set role=presentation if alt attr is missing (#18546)
This applies to all images posted that do not have a user description.
2022-10-12 14:07:37 +03:00
Martin Brennan 8ebd5edd1e
DEV: Rename secure_media to secure_uploads (#18376)
This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality.

This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site.

Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing.

This also keeps compatibility with the `secure-media-uploads` path, and changes new
secure URLs to be `secure-uploads`.

Deprecated settings:

* secure_media -> secure_uploads
* secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails
* secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 09:24:33 +10:00
Keegan George a23d19fab0
DEV: Emoji picker keyboard accessibility updates (#18331)
This PR makes some updates to the prior keyboard accessibility commit (eb98746):
- Makes `tabindex` attribute only appear on emoji markup in the emoji picker.
- After pressing the Esc key, focus returns to the <textarea/> input (composer editor or chat input)
2022-09-22 15:21:34 -07:00
Keegan George eb987460f2
FEATURE: Select emojis in picker via keyboard (#18163)
* DEV: Make emoji elements focusable

Since emoji elements are of type `<img>` it requires a `tablindex="0"` in order to be focusable.

* WIP: Handle emoji focus/selection via arrow keys

Near completion, however, need a few fixes/improvements and overall code cleanup

* WIP: Testing

* DEV: Fixes and cleanup

* DEV: Follow conventions

* DEV: Improve up/down traversal when recents present

* DEV: Emoji markup in tests should include `tabindex`

* DEV: Add `tabindex` to topic tests

* DEV: Variable name as `searchInput` instead of `searchBar`

* DEV: Use appropriate method name (`_setNumEmojiPerRow`)

* DEV: Add comments and avoid nested if

* WIP: Adding test

* Fix first test

* DEV: Add assertions for arrow keys and escape key

* Some fixes for up/down navigation

This does not fix everything, when going from one section to another,
there are issues

* Fix a small regression

* FIX: Ability to focus on search results

Fixes regression

* Refactor calculating next up/down emoji

* Debugging test failure

* Skip stubborn CI test, add others

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2022-09-21 13:21:36 -07:00
David Taylor 4bf9b73296
DEV: Ensure `censorFn` copes with null `regexpList` (#17754)
This fixes the test suite failures introduced by 862007fb18
2022-08-02 11:09:51 +01:00
Selase Krakani 862007fb18
FEATURE: Add support for case-sensitive Watched Words (#17445)
* FEATURE: Add case-sensitivity flag to watched_words

Currently, all watched words are matched case-insensitively. This flag
allows a watched word to be flagged for case-sensitive matching.
To allow allow for backwards compatibility the flag is set to false by
default.

* FEATURE: Support case-sensitive creation of Watched Words via API

Extend admin creation and upload of Watched Words to support case
sensitive flag. This lays the ground work for supporting
case-insensitive matching of Watched Words.

Support for an extra column has also been introduced for the Watched
Words upload CSV file. The new column structure is as follows:

 word,replacement,case_sentive

* FEATURE: Enable case-sensitive matching of Watched Words

WordWatcher's word_matcher_regexp now returns a list of regular
expressions instead of one case-insensitive regular expression.

With the ability to flag a Watched Word as case-sensitive, an action
can have words of both sensitivities.This makes the use of the global
Regexp::IGNORECASE flag added to all words problematic.

To get around platform limitations around the use of subexpression level
switches/flags, a list of regular expressions is returned instead, one for each
case sensitivity.

Word matching has also been updated to use this list of regular expressions
instead of one.

* FEATURE: Use case-sensitive regular expressions for Watched Words

Update Watched Words regular expressions matching and processing to handle
the extra metadata which comes along with the introduction of
case-sensitive Watched Words.

This allows case-sensitive Watched Words to matched as such.

* DEV: Simplify type casting of case-sensitive flag from uploads

Use builtin semantics instead of a custom method for converting
string case flags in uploaded Watched Words to boolean.

* UX: Add case-sensitivity details to Admin Watched Words UI

Update Watched Word form to include a toggle for case-sensitivity.
This also adds support for, case-sensitive testing and matching of  Watched Word
in the admin UI.

* DEV: Code improvements from review feedback

 - Extract watched word regex creation out to a utility function
 - Make JS array presence check more explicit and readable

* DEV: Extract Watched Word regex creation to utility function

Clean-up work from review feedback. Reduce code duplication.

* DEV: Rename word_matcher_regexp to word_matcher_regexp_list

Since a list is returned now instead of a single regular expression,
change `word_matcher_regexp` to `word_matcher_regexp_list` to better communicate
this change.

* DEV:  Incorporate WordWatcher updates from upstream

Resolve conflicts and ensure apply_to_text does not remove non-word characters in matches
that aren't at the beginning of the line.
2022-08-02 10:06:03 +02:00
Andrei Prigorshnev 48e2caf7b7
DEV: emoji helper: add the ability to set custom title (#17517) 2022-07-18 21:51:19 +04:00
Jarek Radosz 057d6b406d
DEV: Extensively use `includes()` (#17541)
Also, the change in insert-hyperlink (from `this.linkUrl.indexOf("http") === -1` to `!this.linkUrl.startsWith("http")`) was intentional fix: we don't want to prevent users from looking up topics with http in their titles.
2022-07-17 20:48:36 +02:00
Jarek Radosz 5f7163b5bb
DEV: Extensively use `startsWith()` (#17540) 2022-07-17 20:16:39 +02:00
Jarek Radosz 5538b8442e
DEV: Introduce `discourseLater` (#17532)
A wrapper for `later()` from `@ember/runloop`, similar to `discourseDebounce`. It automatically reduces the delay in testing environment.
2022-07-17 00:50:49 +02:00
Jarek Radosz c3fd91670e
DEV: Update linting setup and fix issues (#17345)
Re-lands #16119 and #17298

* Update eslint-config-discourse
* Update linting workflow
* Prettier-ignore stuff
* Update template-lint config
* Auto-fix template issues
* Fix various template issues
  Mostly incorrect attributes and unused templates
* Prettier js files
* Fix template auto-fix regressions
* Small css tweak

Co-authored-by: Peter Wagenet <peter.wagenet@gmail.com>
2022-07-06 10:37:54 +02:00
Rafael dos Santos Silva 9833b59b70
FEATURE: Allow iframe `allow` attribute in posts (#17291)
This is used so iframes can use mic, camera, EME, etc.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Feature-Policy#directives
lists current possible values

Feature request https://meta.discourse.org/t/iframe-attributes-not-working/127383?u=falco
2022-07-05 12:25:27 -03:00
Gerhard Schlager 1c6f8f8a36
DEV: Upgrade Markdown.it to v13.0.1 (#17099)
Updates markdown-it to v13.0.1

Noteworthy changes:
* `markdownit()` is now available on `globalThis` instead of `window`.

* The `text_collapse` rule was renamed to `fragments_join` which affected the `bbcode-inline` implementation.

* The `linkify` rule was added to the `inline` chain which affected the handling of the `[url]` BBCode. If available, our implementation reuses `link_open` and `link_close` tokens created by linkify in order to prevent duplicate links.

* The rendered HTML for code changed slightly. There's now a linebreak before the `</code>` tag. The tests were adjusted accordingly.
2022-06-20 15:25:13 +02:00
David Taylor 166fe3bb34
FIX: Apply 'allowed_href_schemes' to all src/srcset attributes (#16860)
Previously we were only applying the restriction to `a[href]` and `img[src]`. This commit ensures we apply the same logic to all allowlisted media src attributes.
2022-05-19 11:18:30 +01:00
Gerhard Schlager 3d62e335f7
FEATURE: Detect emoji from Emoji 14.0 (#16531) 2022-04-22 08:42:15 +02:00
Jarek Radosz 77fed8dfc3
DEV: Update xss.js package (#16398) 2022-04-06 22:49:13 +02:00
Jarek Radosz 06d4445ab0
DEV: Remove deprecated pretty-text functions/file (#16397)
Nothing in all-the* uses those anymore
2022-04-06 22:48:50 +02:00
CommanderRoot 86a783b3ad
DEV: Replace deprecated String.prototype.substr() (#16233)
String.prototype.substr() is deprecated so we replace it with String.prototype.slice() which works similarily but isn't deprecated.

Signed-off-by: Tobias Speicher <rootcommander@gmail.com>
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2022-04-01 17:35:17 +02:00
Jarek Radosz b9c90d6a06
DEV: Use `Map()` to store emoji groups (#16065) 2022-03-05 18:58:49 +01:00
Martin Brennan cfe4ff8d56
FIX: Make sure html_raw is hoisted in custom markdown cook function (#16050)
When returning the customRenderFn from within buildCustomMarkdownCookFunction
for custom markdown engines (such as the one used by the [chat] transcripts)
we were not hoisting/unhoisting the `html_raw` tokens created by the
transcript, which meant that opts.discourse.hoisted could end up in
a state where it was null, and which caused errors and general unpleasantness.

Instead, we can just call the `cook` function that is already exported
from discourse-markdown-it, that takes care of what we did previously
plus the hoisting.

There is a companion chat commit that adds tests for this, there are
no custom markdown engine usages in core to test with.
2022-02-28 07:54:55 +10:00
Martin Brennan 599a72768c
DEV: Add force_quote_link option to PrettyText (#16034)
This option will make it so the [quote] bbcode will always
include the HTML link to the quoted post, even if a topic_id
is not provided in the PrettyText#cook options. This is so
[quote] bbcode can be used in other places, like chat messages,
that always need the link and do not have an "off-topic" ID
to use.
2022-02-23 16:13:46 +10:00
Bianca Nenciu add4b74e08
FIX: Load short upload URLs only once (#15918)
Loading did not work when it was used for multiple posts. Only the
short URLs from the first post were loaded.
2022-02-16 08:57:20 +02:00
Joffrey JAFFEUX b3ecf00c98
DEV: adds initial support for custom blocks using code fencing (#15743)
Allows to write custom code blocks:

```
   ```mermaid height=200,foo=bar
   test
   ```
```

Which will then get converted to:

```
<pre data-code-wrap="mermaid" data-code-height="200" data-code-foo="bar">
  <code class="lang-nohighlight">
    test
  </code>
</pre>
```
2022-02-09 11:23:44 +01:00
Martin Brennan f3467ee673
FEATURE: Add requestCustomMarkdownCookFunction API (#15753)
This commit adds a requestCustomMarkdownCookFunction function
to the `helper` that is provided to custom markdown rules
via their `setup` function.

The way this works is that once the default markdown engine that
we use for cooking posts has been set up, we loop through all
of the callbacks registered by `requestCustomMarkdownCookFunction`
and call `_buildCustomMarkdownCookFunction`. This creates
a new markdown engine using many of the same settings as the
default one, but will allow for the following options to be
changed by the markdown rule requesting the custom function:

* featuresOverride - The markdown-it features to allow for the engine
* markdownItRules - The markdown-it rules to allow for the engine

After this engine is set up a render function which renders + sanitizes
the output is returned for use by the markdown rule.

The use case for this API is mainly for block BBCode markdown rules
which want to render their content with a limited subset of the
markdown features/rules. Our initial use case for this is chat message
quoting.

This commit also does some minor refactoring of discourse-markdown-it
to accommodate this new engine building.
2022-02-04 09:00:40 +10:00
Jarek Radosz 6f8c91254e
FEATURE: Partial match aliases in emoji filter (#15613) 2022-02-01 22:36:48 +01:00
Martin Brennan 0b8d0a14d7
DEV: Add markdown_additional_options to Site (#15738)
Sometimes plugins need to have additional data or options available
when rendering custom markdown features/rules that are not available
on the default opts.discourse object. These additional options should
be namespaced to the plugin adding them.

```
Site.markdown_additional_options["chat"] = { limited_pretty_text_markdown_rules: [] }
```

These are passed down to markdown rules on opts.discourse.additionalOptions.

The main motivation for adding this is the chat plugin, which currently stores
chat_pretty_text_features and chat_pretty_text_markdown_rules on
the Site object via additions to the serializer, and the Site object is
not accessible to import via markdown rules (either through
Site.current() or through container.lookup). So, to have this working
for both front + backend code, we need to attach these additional options
from the Site object onto the markdown options object.
2022-01-28 13:02:02 +10:00
Rafael dos Santos Silva 8cfd791e0f
FEATURE: Add missing emojis (#15582)
* FEATURE: Add missing emojis

* More missing emojis
2022-01-14 17:51:13 -03:00
Natalie Tay dd3ed27930
DEV: Allow utility class domFromString to take in strings with multiple top level elements (#15548)
Previously only `<div>one top element</div>` was allowed because we use `firstChild` instead of `children`.
We also want `<div>one</div><div>two</div>` to work with this method.
2022-01-12 19:49:24 +08:00
Alan Guo Xiang Tan c2afc3915b FEATURE: Customizable rules and plugins for `PrettyText.markdown`.
This commit extends the options which can be passed to
`PrettyText.markdown` so that which Markdown-it rules and Discourse
Markdown plugins to be used when rendering a text can be customizable.
Currently, this extension is mainly used by plugins.
2022-01-11 10:39:40 +08:00
Jarek Radosz 2dcad9772f FIX: Don't display duplicated emoji in picker 2022-01-11 10:37:02 +08:00
Rafael dos Santos Silva 06bba76719
DEV: Update emojis constants (#15506)
Follow up to d1cfabd so the new emojis are usable by the app
2022-01-10 14:53:52 -03:00
Alan Guo Xiang Tan bdc868246f DEV: Removing unused lines of code.
The code had no effect on how the markdown-it is being configured.
2022-01-07 09:06:16 +08:00
Alan Guo Xiang Tan 85e124bae1 DEV: Remove redundant lines of code.
Engines are enabled by default so there is no need to specify Discourse
specific engines in the object.
2022-01-06 09:02:52 +08:00
Joffrey JAFFEUX cb976ac562
DEV: creates domFromString utility function (#15310) 2021-12-17 09:25:34 +01:00
Penar Musaraj 60851037d8
DEV: Remove unused composer editor preview sync (#15322) 2021-12-16 17:18:54 -05:00
Jarek Radosz 6e8df3d66b
DEV: Deprecate `registerOption()` (#15209)
It's been a no-op for 5 years now.
2021-12-07 18:33:45 +01:00
Bianca Nenciu 148ee1d162
FIX: Do not perform link lookup for replaced links (#14742)
A link that was added because a watched word was replaced could create
a notice if the same link was present before.
2021-10-28 13:27:31 +03:00
Joffrey JAFFEUX 76a9ca99a8
DEV: removes jquery usage from onebox (#14683) 2021-10-22 13:15:46 +02:00
Qursch 11763a5e8d
FEATURE: Add reversed and type to allowed attributes (#14626) 2021-10-18 12:11:13 +11:00
Bianca Nenciu ba81d1853b
FIX: Disable previews if diffhtml is enabled (#14537)
diffhtml should not rerender video and audio elements so there is no
point in having these.
2021-10-08 15:57:08 +03:00
Bianca Nenciu 38199424bc
SECURITY: Sanitize d-popover attributes (#13958) 2021-08-05 16:39:17 +03:00