We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)
This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes
1. We attempt to localize a string on each site, ensuring we warmup
the i18n
2. We preload all our templates (compiling .erb to class)
3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).
Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
This commit introduces 2 features:
1. DISCOURSE_COMPRESS_ANON_CACHE (true|false, default false): this allows
you to optionally compress the anon cache body entries in Redis, can be
useful for high load sites with Redis that lives on a separate server to
to webs
2. DISCOURSE_ANON_CACHE_STORE_THRESHOLD (default 2), only pop entries into
redis if we observe them more than N times. This avoids situations where
a crawler can walk a big pile of topics and store them all in Redis never
to be used. Our default anon cache time for topics is only 60 seconds. Anon
cache is in place to avoid the "slashdot" effect where a single topic is
hit by 100s of people in one minute.
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.
Customizations can be added and edited in Admin > Customize > Email Style.
Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.
As part of this work, RTL locales are now rendered correctly for all emails.
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""
This reverts commit f89bd55576.
* Replace .tar.zip with .zip
* FEATURE: admin/user exports are compressed using the zip format
* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files
* Fix controller test, updated locale and skip saving the csv export to disk
Overcommit uses prebuilt hooks and require global installation.
To avoid this issues replace it with Lefthook.
Lefthook will be installed with npm packages. New contributors
will have fully consistent git hooks.
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.
Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
- Update github_blob engine to support displaying stl files
- FEATURE: add `data-original-href` attribute to Vimeo iframes
- Add poster image for video oneboxes
Adds the parallel_tests gem, and redis/postgres configuration for running rspec tests in parallel. To use:
```
rake parallel:rake[db:create]
rake parallel:rake[db:migrate]
rake parallel:spec
```
This brings the test suite from 12m20s to 3m11s on my macOS machine
Improves the generic database used by some import scripts:
* Adds additional columns for users
* Adds support for attachments
* Allows setting the data type for keys (numeric or string) to ensure correct sorting
Two very low risk updates, message_bus has been released no need to depend
on pre-release.
mini_racer update is for a very minor change (shared isolates are not used
in discourse so it is not a fix we technically need)
Launching Discourse no longer should require foreman in dev. We can simply
use `bin/unicorn` which automatically launches a sidekiq worker.
The foreman gem depends on thor ~> 0.19.1 which is no longer supported in
rails 6. So this pre-emptively prepares us for it.
Message bus uses a key to keep track of the last id for each channel
this key was never expired even if channel data expired
This change ensures we also expire the tracking key, it means a lot to us
cause each topic has a channel, so if you have 1 million topics that is
1 million keys that may persist forever
* FEATURE: Added MaxMindDb to resolve IP information.
* FEATURE: Added browser detection based on user agent.
* FEATURE: Added recently used devices in user preferences.
* DEV: Added acceptance test for recently used devices.
* UX: Do not show 'Show more' button if there aren't more tokens.
* DEV: Fix unit tests.
* DEV: Make changes after code review.
* Add more detailed unit tests.
* Improve logging messages.
* Minor coding style fixes.
* DEV: Use DropdownSelectBoxComponent and run Prettier.
* DEV: Fix unit tests.
Introduce new patterns for direct sql that are safe and fast.
MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API
- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder
See more at: https://github.com/discourse/mini_sql
This updates tests to use latest rails 5 practice
and updates ALL dependencies that could be updated
Performance testing shows that performance has not regressed
if anything it is marginally faster now.
* Feature: Push notifications for Android
Notification config for desktop and mobile are merged.
Desktop notifications stay as they are for desktop views.
If mobile mode, push notifications are enabled.
Added push notification subscriptions in their own table, rather than through
custom fields.
Notification banner prompts appear for both mobile and desktop when enabled.
* Use a EmailValidator.email_regexp for `Email.is_valid?`
check as we're seeing an increase in allocation when
parsing email addresses wih `Mail::Address`.
implemented review items.
Blocking previous codes - valid 2-factor auth tokens can only be authenticated once/30 seconds.
I played with updating the “last used” any time the token was attempted but that seemed to be overkill, and frustrating as to why a token would fail.
Translatable texts.
Move second factor logic to a helper class.
Move second factor specific controller endpoints to its own controller.
Move serialization logic for 2-factor details in admin user views.
Add a login ember component for de-duplication
Fix up code formatting
Change verbiage of google authenticator
add controller tests:
second factor controller tests
change email tests
change password tests
admin login tests
add qunit tests - password reset, preferences
fix: check for 2factor on change email controller
fix: email controller - only show second factor errors on attempt
fix: check against 'true' to enable second factor.
Add modal for explaining what 2fa with links to Google Authenticator/FreeOTP
add two factor to email signin link
rate limit if second factor token present
add rate limiter test for second factor attempts
oga gem is automatically required by the aws gem
the oga gem retains about 1mb of memory, aws now uses nokogiri
This also removes the html normalize from the pretty text specs that was
a fair bit buggy as the polls test shows.
* SPEC: PollFeedJob parsing atom feed
* add FeedItemAccessor
It is to provide a consistent interface to access a feed item's tag
content.
* add FeedElementInstaller
to install non-standard and non-namespaced feed elements
* FEATURE: replace SimpleRSS with Ruby RSS module
* get FinalDestination and download with Excon
* support namespaced element with FeedElementInstaller
This adds the markdown.it engine to Discourse.
https://github.com/markdown-it/markdown-it
As the migration is going to take a while the new engine is default
disabled. To enable it you must change the hidden site setting:
enable_experimental_markdown_it.
This commit is a squash of many other commits, it also includes some
improvements to autospec (ability to run plugins), and a dev dependency
on the og gem for html normalization.
This feature introduces the concept of themes. Themes are an evolution
of site customizations.
Themes introduce two very big conceptual changes:
- A theme may include other "child themes", children can include grand
children and so on.
- A theme may specify a color scheme
The change does away with the idea of "enabled" color schemes.
It also adds a bunch of big niceties like
- You can source a theme from a git repo
- History for themes is much improved
- You can only have a single enabled theme. Themes can be selected by
users, if you opt for it.
On a technical level this change comes with a whole bunch of goodies
- All CSS is now compiled using a custom pipeline that uses libsass
see /lib/stylesheet
- There is a single pipeline for css compilation (in the past we used
one for customizations and another one for the rest of the app
- The stylesheet pipeline is now divorced of sprockets, there is no
reliance on sprockets for CSS bundling
- CSS is generated with source maps everywhere (including themes) this
makes debugging much easier
- Our "live reloader" is smarter and avoid a flash of unstyled content
we run a file watcher in "puma" in dev so you no longer need to run
rake autospec to watch for CSS changes
Rails yanked out observers many many years ago, instead the functionality
was yanked out to a gem that is very lightly maintained.
For example: if we want to upgrade to rails 5 there is no published gem
Internally the usage of observers had quite a few problem.
The series of refactors renamed a bunch of classes to give us more clarity
and removed some magic.
Sam Saffron
Martin Brennan
Krzysztof Kotlarek
Arpit Jalan
Guo Xiang Tan
Neil Lalonde
Roman Rizzi
Joffrey JAFFEUX
Abroskin Alexander
David Taylor
Maja Komel
Daniel Waterworth
Nicolas Sebastian Vidal
Gerhard Schlager
Vinoth Kannan
Régis Hanol
Penar Musaraj
Bianca Nenciu
Jeff Wong
Rafael dos Santos Silva
Ori Pekelman
Jano Suchal
Kyle Zhao
Bianca Nenciu
Erick Guan
Guo Xiang Tan
Robin Ward