Commit Graph

81 Commits

Author SHA1 Message Date
Joffrey JAFFEUX 6493ddce17
PERF: limits use of redis cache while building emojis list (#19013)
We were doing get on Redis two times for each emoji while building the custom/standard/all lists which where resulting in ~3710 Redis calls. Given the emoji DB file is loaded in memory while we build/cache the emojis list this is unnecessary and slow.

As a simplification in pseudo code here is an explanation of what we were doing:

```ruby
emojis.each |emoji_name|
  aliases = get_aliases_from_redis_cache(emoji_name)
  is_tonable = get_is_tonable_from_redis_cache(emoji_name)
  build_emoji(emoji_name, aliases, is_tonable)
end
```

The two redis calls are now simplified to a simple hash access: `@db[emoji_name]`
2022-11-14 13:38:50 +01:00
Joffrey JAFFEUX a705e4815f
FIX: do not include group less emojis in standard list (#18659) 2022-10-19 09:53:56 +02:00
Joffrey JAFFEUX 3bd94b2794
DEV: includes search_aliases in emoji object (#18608)
This is an information we need on client side for example when filtering emojis.
2022-10-15 07:09:00 -05:00
Joffrey JAFFEUX 09a434e2d8
DEV: emoji model should set the group of the emoji (#18237)
We do it for custom emojis but not for standard emojis which is quite unexpected and limit the use cases we can have for it.
2022-09-14 13:10:48 +02:00
Sam 33a0ad1b69
PERF: introduce site/global emoji cache (#15899)
Previously calls such as `Emoji["smile"]` would force a full dehydration of
objects from Redis.

This introduces a version safe site and global emoji cache so lookups are
cheap. It eliminates iterating through the list of emojis and pulling from
redis.

Distributed cache uses a normalized name as the key and stores an Array tuple
with version and Emoji. Successful hits always confirm version matches.

Interface to Emoji object remains unchanged.

We opted for 2 caches to improve reuse on multisites. misses though will be
stored in both caches. If there is a hit on the global cache we can avoid
looking up in site local cache and storing a miss there.
2022-02-16 12:46:17 +11:00
Ayke Halder 5ff3a9c4bb
DEV: add native lazy loading for emojis (#15830) 2022-02-09 12:18:59 +01:00
Rafael dos Santos Silva 8cfd791e0f
FEATURE: Add missing emojis (#15582)
* FEATURE: Add missing emojis

* More missing emojis
2022-01-14 17:51:13 -03:00
Rafael dos Santos Silva 06bba76719
DEV: Update emojis constants (#15506)
Follow up to d1cfabd so the new emojis are usable by the app
2022-01-10 14:53:52 -03:00
Joffrey JAFFEUX 5eb6e9281a
FIX: manually adds frowning_face_with_open_mouth for apple (#13528) 2021-07-21 23:27:20 +02:00
Roman Rizzi fa4e5e8dad
FEATURE: Render emojis on GitHub labels when oneboxing an issue. (#13531) 2021-06-25 14:48:36 -03:00
Rafael dos Santos Silva 83f332b5a5
FEATURE: Add a site setting to allow emojis to come from an external URL (#12180) 2021-03-02 16:04:16 -03:00
Joffrey JAFFEUX 54519a2976
FIX: do not replace smiling_face by slight_smile (#11412) 2020-12-06 18:28:39 +01:00
Daniel Waterworth 721ee36425
Replace `base_uri` with `base_path` (#10879)
DEV: Replace instances of Discourse.base_uri with Discourse.base_path

This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Joffrey JAFFEUX 77801aa9be
FIX: allows to have custom emoji translation without static file (#9893) 2020-05-27 20:11:52 +02:00
Joffrey JAFFEUX 1d685c22af
REVERT: removes translate_emoji (#9889)
This API is actually used in some plugins.
2020-05-27 12:08:24 +02:00
Robin Ward b2e4ca968b DEV: Remove `translate` API from plugin.
It appears to be completely unused at this point.
2020-05-08 14:14:01 -04:00
Sam Saffron d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
Joffrey JAFFEUX 0996c3b7b3
FEATURE: allows multiple custom emoji groups (#9308)
Note: DBHelper would fail with a sql syntax error on columns like "group".

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2020-03-30 20:16:10 +02:00
Sam Saffron e9cf0de588 FEATURE: do not replace &harr; with an emoji
&harr; technically can be an emoji, but usually will not be used in that way
2019-08-30 15:06:23 +10:00
Joffrey JAFFEUX 5d72561f4f
FIX: allows replacement of digits and symbols emojis (#7978) 2019-08-07 11:38:58 +02:00
Joffrey JAFFEUX 9ee2c121c1
FIX: ensures emoji helper is working with custom emojis (#7843) 2019-07-03 09:23:40 +02:00
Joffrey JAFFEUX dfde59f631
FIX: provides an emoji helper to replace codes by images (#7802) 2019-06-26 15:11:04 +02:00
Sam fa2a5f6f56
FEATURE: SKIP_DB_AND_REDIS env var (#7756)
Sometimes we would like to create a base image without any DB access, this
assists in creating custom base images with custom plugins that already
includes `public/assets`

Following this change set you can run:

```
SPROCKETS_CONCURRENT=1 DONT_PRECOMPILE_CSS=1 SKIP_DB_AND_REDIS=1 RAILS_ENV=production bin/rake assets:precompile
```

Then it is straight forward to create a base image without needing a DB or
Redis.
2019-06-13 12:58:27 +10:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Tim Lange f7b156ffbd UX: Better emoji escaping for topic title (#7218)
* FIX: Fixed failing discourse-prometheus-alert-receiver plugin specs
2019-03-21 09:11:33 +01:00
Régis Hanol 6fb49e74a1
Revert "UX: Better emoji escaping for topic title (#7176)" (#7201)
This reverts commit 0d9bc0aaa6.
2019-03-19 12:02:47 +01:00
Tim Lange 0d9bc0aaa6 UX: Better emoji escaping for topic title (#7176) 2019-03-19 09:33:10 +01:00
Joffrey JAFFEUX d91b47064e
FIX: windows has no country flags (#7199) 2019-03-18 19:28:49 +01:00
Joffrey JAFFEUX 0a8f950281
FIX: bumps emoji cache version (#7195)
Might not be necessary, but might avoid weird results with all the recents emoji changes.
2019-03-18 08:22:36 +01:00
Joffrey JAFFEUX aa4b84925a FEATURE: updates emoji to latest 12.0 version (#7095)
* improved emoji support

- always optimize images as part of the task
- use the unicode standard ordering/naming for sections

* UX: more height for when there are recently used
2019-03-15 16:12:07 +01:00
Joffrey JAFFEUX d32557ea32 Revert "FIX: Better emoji escaping for topic title"
This reverts commit 35426b5ad6.
2019-03-13 13:02:56 +01:00
Tim Lange 35426b5ad6 FIX: Better emoji escaping for topic title
This commit also puts emojiVersion in its own erb file.
2019-03-13 11:17:59 +01:00
Régis Hanol 95e5f8380d FEATURE: Allow plugins to add custom emoji translations
FIX: buildTranslationTree was erroring when translations overlapped (ie. ":-)" and ":-))")
FIX: emoji translations wasn't working properly when translations overlapped
2019-01-04 15:27:46 +01:00
Joffrey JAFFEUX 3fbf7b164d
FEATURE: emoji 11.0 "recently added" update (#6275)
Adds support for emojis in the http://www.unicode.org/emoji/charts-11.0/emoji-released.html list
2018-08-16 11:59:45 +02:00
Joffrey JAFFEUX 691174fa8f
FEATURE: adds emoji search aliases 2018-05-01 15:43:49 +02:00
Régis Hanol 2d48caffdf FIX: be more lenient when deleting a custom emoji 2017-11-20 23:50:23 +01:00
Guo Xiang Tan 23b787e0a6 Require dependency otherwise it causes Sidekiq to lock up in development. 2017-09-25 13:48:59 +08:00
Guo Xiang Tan f496302c2c `order` scope is ignored when using `find_each`. 2017-08-07 11:46:18 +09:00
Régis Hanol c26ba890bd prevent N+1 when loading custom emojis 2017-08-04 17:04:59 +02:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Robin Ward 574681dc47 FIX: Show emoji in inline oneboxes 2017-07-21 14:24:48 -04:00
Joffrey JAFFEUX 54e0075a3e use the EMOJI_VERSION constant 2017-07-21 19:42:49 +02:00
Joffrey JAFFEUX 6de258d4cf FEATURE: Introduces new emoji-picker 2017-07-19 16:48:23 +02:00
Guo Xiang Tan fa35137d3a PERF: Reduce allocations in `Emoji.unicode_replacements`.
```
MemoryProfiler.report { Emoji.unicode_replacements }.pretty_print
```

Before:

```
Total allocated: 2032131 bytes (30219 objects)
Total retained:  798059 bytes (10734 objects)

allocated memory by gem
-----------------------------------
   1270762  discourse/app
    690149  2.4.1/lib
     48764  activesupport-4.2.8
     11405  bootsnap-1.0.0
      5266  logster-1.2.7
      2473  hiredis-0.6.1
      2056  redis-3.3.3
       840  discourse/lib
       240  activerecord-4.2.8
       176  thread_safe-0.3.6
```

After:

```
Total allocated: 1715713 bytes (22449 objects)
Total retained:  797985 bytes (10737 objects)

allocated memory by gem
-----------------------------------
    957250  discourse/app
    690028  2.4.1/lib
     48764  activesupport-4.2.8
     11445  bootsnap-1.0.0
      2745  hiredis-0.6.1
      2624  redis-3.3.3
      1473  logster-1.2.7
       928  discourse/lib
       280  activerecord-4.2.8
       176  thread_safe-0.3.6
```
2017-06-20 13:50:37 +09:00
Guo Xiang Tan 40ef5c4f14 PERF: Reduce memory allocation by `Emoji.unicode_replacements`.
Calling `Emoji.tonable_emojis` fetches from the cache but creates
a giant array each time it is called.

```
MemoryProfiler.report { Emoji.unicode_replacements }.pretty_print
```

Before:

```
Total allocated: 16560770 bytes (246364 objects)
Total retained:  784793 bytes (10624 objects)

allocated memory by gem
-----------------------------------
  10224581  activesupport-4.2.8
   3504241  hiredis-0.6.1
   1449466  discourse/app
    679314  2.4.1/lib
    419592  discourse/lib
    226720  redis-3.3.3
     56680  activerecord-4.2.8
       176  thread_safe-0.3.6
```

After:

```
Total allocated: 2127703 bytes (30556 objects)
Total retained:  798099 bytes (10735 objects)

allocated memory by gem
-----------------------------------
   1270762  discourse/app
    690149  2.4.1/lib
    106977  bootsnap-1.0.0
     48764  activesupport-4.2.8
      5266  logster-1.2.7
      2473  hiredis-0.6.1
      2056  redis-3.3.3
       840  discourse/lib
       240  activerecord-4.2.8
       176  thread_safe-0.3.6
```
2017-06-20 13:10:44 +09:00
Régis Hanol f77e6c8fc6 Merge pull request #4928 from jjaffeux/transform-native-emojis-to-names
Transforms native emojis to valid names
2017-06-14 15:43:09 +02:00
Joffrey JAFFEUX ab6086bce9 Transforms native skin toned emojis to valid names 2017-06-14 15:35:37 +02:00
Régis Hanol 04e175bc06 Merge pull request #4925 from jjaffeux/adds-tones-to-clear-cache
Makes sure we clear tonable_emojis cache
2017-06-14 14:56:41 +02:00
Joffrey JAFFEUX fa508042ff Makes sure we clear tonable_emojis cache 2017-06-14 11:11:02 +02:00
Joffrey JAFFEUX 299339a373 Fixes emoji name to code missing tones 2017-06-13 14:10:26 -04:00