Commit Graph

192 Commits

Author SHA1 Message Date
Sam Saffron ae2a56999e Revert "FIX: update Redis gem to version 4.1.3 (#8197)"
This reverts commit ab74a50d85.

We really want to upgrade redis, but discovered some edge cases
around failover we need to test.

Holding off on the upgrade till a bit more testing happens
2019-10-17 11:41:46 +11:00
Krzysztof Kotlarek ab74a50d85 FIX: update Redis gem to version 4.1.3 (#8197)
* FIX: update Redis gem to version 4.1.3

I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

* FIX: get rid of redis freedom patch
2019-10-17 08:49:23 +11:00
Sam Saffron 586dfcc795 PERF: avoid regex for hot path
Camelize is called quite a lot in zeitwerk, avoid using a regex here which
is far slower than using ends_with?
2019-10-08 13:43:54 +11:00
Krzysztof Kotlarek 35b1185a08 FIX: Revert Demon::DemonBase back to Demon::Base (#8132)
I introduced DemonBase because I had got some conflict between `demon/base.rb` and `jobs/base.rb`, however, to not rename base class, it is possible to use regex on absolute path in Zeitwerk custom inflector.
2019-10-02 14:54:08 +10:00
Krzysztof Kotlarek 427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Sam Saffron 445d305154 DEV: initial migration can fail
db:migrate can issue translations due to module loading localizing
end user messages

This allows db:migrate to work even when db is blank
2019-09-17 13:38:01 +10:00
Sam Saffron 082f59842d DEV: bind for thread local vars should yield block
followup on #015051ec without this improvement monkey patch does not work.

bind should yield the block it is passed for it to work.
2019-09-12 17:57:57 +10:00
Sam Saffron 015051ecaf PERF: avoid spinning a thread each time we close a connection
This is a temporary workaround for the issue in https://github.com/rails/rails/pull/36949

Discussing a proper fix in Rails with the Rails team.

Prior to this fix we were spinning up a thread every time we closed a connection
to the db.
2019-09-12 17:34:04 +10:00
Krzysztof Kotlarek 32b8a2ccff DEV: Upgrade Discourse to Rails 6 (#8083)
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
2019-09-12 10:41:50 +10:00
Gerhard Schlager 24877a7b8c FIX: Correctly encode non-ASCII filenames in HTTP header
Backport of fix from Rails 6: 890485cfce
2019-08-07 19:10:50 +02:00
Sam fa2a5f6f56
FEATURE: SKIP_DB_AND_REDIS env var (#7756)
Sometimes we would like to create a base image without any DB access, this
assists in creating custom base images with custom plugins that already
includes `public/assets`

Following this change set you can run:

```
SPROCKETS_CONCURRENT=1 DONT_PRECOMPILE_CSS=1 SKIP_DB_AND_REDIS=1 RAILS_ENV=production bin/rake assets:precompile
```

Then it is straight forward to create a base image without needing a DB or
Redis.
2019-06-13 12:58:27 +10:00
Gerhard Schlager a77d8bee68 FIX: Use correct locale when translating without cache
follow-up to 6d8eb9c1
2019-06-05 14:19:56 +02:00
Sam Saffron 6d8eb9c144 PERF: bypass slow locale lookups in various cases
Previously as soon as any override was defined we would regress to the slow
path for locale lookups. Additionally if `raise: true` was specified which
rails likes to add in views we would bypass the cache

The new design manages to use the fast path for many more cases
2019-06-05 14:30:25 +10:00
Gerhard Schlager a3e4f43a4d DEV: Remove patch for I18n.exists?
It was merged upstream a long time ago
2019-05-13 09:36:05 +02:00
Sam Saffron 982c4496ee Correct some missing spots for frozen_string_literal 2019-05-13 09:31:32 +08:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron e2bcf55077 DEV: move send => public_send in lib folder
This handles most of the cases in `lib` where we were using send instead
of public_send
2019-05-07 12:25:44 +10:00
Guo Xiang Tan 152238b4cf DEV: Prefer `public_send` over `send`. 2019-05-07 09:33:21 +08:00
Sam Saffron 1be01f8dd4 DEV: Add support for Rails 6
Minor fixes to add Rails 6 support to Discourse, we now will boot
with RAILS_MASTER=1, all specs pass

Only one tiny deprecation left

Largest change was the way ActiveModel:Errors changed interface a
bit but there is a simple backwards compat way of working it
2019-05-02 16:23:25 +10:00
sau226 24b3494660 HTTPSify links (#7046) 2019-02-22 16:29:27 +01:00
Sam Saffron a72ed6278b DEV: include comment explaining monkey patch
The monkey patch here is very surprising, make a comment explaining why
it is used in the code.

see: https://github.com/rails/rails/issues/35059 for upstream tracking
2019-01-26 07:25:37 +02:00
Gerhard Schlager 7c4d4331bc FEATURE: Better handling of quotation marks in site text search
It also matches 3 dots with the ellipsis symbol.
2018-11-12 13:26:41 +01:00
Sam bc7c1e411c DEV: Ruby 2.5.1 is no longer supported with Discourse
This removes a monkey patch we no longer need since our containers require
2.5.2 or up for all Discourse installs.

If you are looking to deploy on 2.5.1 which is highly not recommended you
will need to figure out how to apply this diff.
2018-11-09 18:24:58 +11:00
Gerhard Schlager e2770bc1c4 FIX: async reload of locales could result in missing translations 2018-10-01 17:14:36 +02:00
Kyle Zhao f1cb431968 FIX: rescue ActiveRecord::RecordInvalid in find_or_create_by_safe! (#6385)
AR uniqueness validation could raise ActiveRecord::RecordInvalid
2018-09-12 11:03:12 +10:00
Guo Xiang Tan 8dc1463ab3 Enable `Lint/ShadowingOuterLocalVariable` for Rubocop. 2018-09-04 10:16:42 +08:00
Maja Komel 182d9a4666 FIX: escape regex chars when searching site texts 2018-09-02 17:25:57 +10:00
Sam 6ee352510a Correctly handle exception 2018-08-23 16:27:00 +10:00
Sam 57d04200a1 Add Rails 6 note here 2018-08-21 11:09:10 +10:00
Sam ca5a6f0a9d FIX: handle concurrently creating post reply keys
In some very rare conditions this would be called concurrently and fail
2018-08-21 10:59:18 +10:00
Sam b6e9c734f2 replace error with message
So Discourse continues to work on Ruby trunk
2018-07-04 09:42:38 +10:00
Sam 2f0e73f2d6 DEV: fast pluck to use type_map in mini_sql 2018-06-20 17:53:49 +10:00
Sam 44091f20c6 DEV: allow for method deprecation using Discourse.deprecate
New method deprecator will ensure one log message an hour happens
for all deprecated method calls per call site

Also removes unused monkey patches to ActiveRecord::Base
2018-06-20 17:53:49 +10:00
Guo Xiang Tan 6e97242bb9 Monkey patch in 7830a950ef 2018-06-19 09:45:48 +08:00
Sam 87fabdc2f3 FIX: correct pool reaper
This removes a freedom patch and replaces with a custom reaper thread
it also captures an issue where reaper would fail when connections where
empty
2018-06-14 18:22:02 +10:00
Sam 71aa20bd30 FIX: pool drainer to use Rails 5.2 implementation
old implementation did not reap abandoned connections
2018-06-14 15:54:48 +10:00
Sam 66982c7800 FIX: stop using Rails connection reaper in multisite
The Rails 5.2 connection reaper appears to be leaking threads
this is a quick fix to stop it, though we need to make sure we
never leak connection pools as well.
2018-06-14 12:49:30 +10:00
Guo Xiang Tan 139d0813b4 Re-enable fast_pluck freedom patch. 2018-06-13 09:25:23 +08:00
Guo Xiang Tan 29c1f01494 Fix broken pool drainer after the upgrade.
See 6cd5cc375a/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb (L467)

We think there is a leak at the moment because a discarded pool
belonging to the parent process shouldn't be present in the
forked process.
2018-06-08 13:19:07 +08:00
Sam 42847252a4 remove conditional code 2018-06-08 13:38:25 +10:00
Sam 1834417e2f downgrade rails properly 2018-06-07 20:38:22 +10:00
Guo Xiang Tan ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Sam 89ad2b5900 DEV: Rails 5.2 upgrade and global gem upgrade
This updates tests to use latest rails 5 practice
and updates ALL dependencies that could be updated

Performance testing shows that performance has not regressed
if anything it is marginally faster now.
2018-06-07 14:21:33 +10:00
Sam b7037808bb PERF: stop using exec which is blocking for direct sql 2018-05-03 15:50:30 +10:00
Sam 6a3c8fe69c FEATURE: protect against accidental column or table drops
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.

The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
2018-03-21 15:43:32 +11:00
Gerhard Schlager eb52c5469e FEATURE: Allow plugins to register a new locale 2018-01-25 14:57:41 +01:00
Sam 081959227d FIX: unicode titles missing when visiting topic from topic list 2017-12-21 15:20:47 +11:00
Guo Xiang Tan c390169b71 Move monkey patch into freedom_patchers. 2017-11-21 14:15:11 +08:00
Guo Xiang Tan 9d6449ae92 Revert "Guard `ActionRecord::Base.exec_sql` against a readonly PostgreSQL cluster."
This reverts commit c2a5e603c2.
2017-10-19 15:39:03 +08:00
Guo Xiang Tan c2a5e603c2 Guard `ActionRecord::Base.exec_sql` against a readonly PostgreSQL cluster. 2017-10-19 14:47:29 +08:00
Sam f6fdc1ebe8 FEATURE: flexible crawler detection
You can use the crawler user agents site setting to amend what user agents
are considered crawlers based on a string match in the user agent

Also improves performance of crawler detection slightly
2017-09-29 12:31:50 +10:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan a4399c2eab FIX: Translation overrides were not cached by locale properly. 2017-07-07 12:28:00 +09:00
Robin Ward 754c1e5438 Upgrade to Babel 6 2017-07-05 15:25:23 -04:00
Guo Xiang Tan 52e654b3ac FIX: Translation fallback was not using fallback's translation override.
https://meta.discourse.org/t/discobot-falling-back-to-site-locale-is-not-using-customized-copies/65140
2017-07-03 14:52:27 +09:00
Sam a3e8c3cd7b FEATURE: Native theme support
This feature introduces the concept of themes. Themes are an evolution
of site customizations.

Themes introduce two very big conceptual changes:

- A theme may include other "child themes", children can include grand
children and so on.

- A theme may specify a color scheme

The change does away with the idea of "enabled" color schemes.

It also adds a bunch of big niceties like

- You can source a theme from a git repo

- History for themes is much improved

- You can only have a single enabled theme. Themes can be selected by
    users, if you opt for it.

On a technical level this change comes with a whole bunch of goodies

- All CSS is now compiled using a custom pipeline that uses libsass
    see /lib/stylesheet

- There is a single pipeline for css compilation (in the past we used
    one for customizations and another one for the rest of the app

- The stylesheet pipeline is now divorced of sprockets, there is no
   reliance on sprockets for CSS bundling

- CSS is generated with source maps everywhere (including themes) this
    makes debugging much easier

- Our "live reloader" is smarter and avoid a flash of unstyled content
   we run a file watcher in "puma" in dev so you no longer need to run
   rake autospec to watch for CSS changes
2017-04-12 10:53:49 -04:00
Régis Hanol a2c04be718 FIX: eradicate I18n fallback issues 💣
FIX: client's translation overrides were not working when the current locale was missing a key
FIX: ExtraLocalesController.show was not properly handling multiple translations
FIX: JsLocaleHelper#output_locale was not properly handling multiple translations

FIX: ExtraLocalesController.show's spec which was randomly failing
FIX: JsLocaleHelper#output_locale was muting cached translations hashes

REFACTOR: move 'enableVerboseLocalization' to the 'localization' initializer
REFACTOR: remove unused I18n.js methods (getFallbacks, localize, parseDate, toTime, strftime, toCurrency, toPercentage)
REFACTOR: remove all I18n.pluralizationRules and instead use MessageFormat's pluralization rules

TEST: add tests for localization initializer
TEST: add tests for I18n.js
2017-02-24 11:31:21 +01:00
Sam d28d8a1f85 FIX: order by op_likes leads to broken browsing 2016-12-27 19:08:54 +11:00
Robin Ward b601120b39 Support for Raw Handlebars and helpers in Ember 1.13 2016-10-26 10:27:57 -04:00
Robin Ward be1d74d207 Split out common functions into `discourse-common` module 2016-09-22 09:52:19 -04:00
Régis Hanol 038eb6f645 FIX: translations with a symbol as key should also be overridable 2016-08-24 11:53:03 +02:00
Robin Ward bba0fd0654 REFACTOR: PreloadStore to ES6 2016-07-11 12:57:05 -04:00
Robin Ward 7ff5b228cd REFACTOR: Raw Handlebars ported to ES6 2016-07-11 12:57:05 -04:00
Sam d9bcf85edc PERF: make development tolerable under sprockets 3 2016-05-13 18:30:03 +10:00
Guo Xiang Tan 21df320f71
FIX: Duplicated file. 2016-04-18 11:18:37 +08:00
Guo Xiang Tan 256d7a00e9 Update sprockets. (#4167)
* Update sass-rails.

* FIX: Tilt dependency has been removed from Ember::Handlebars::Template.

* Update `DiscourseIIFE` to new Sprockets API.

* `Rails.application.assets` returns `nil` in production.

* Move sprockets-rails out of the assets group.

* Pin ember-rails to 0.18.5 which works with Sprockets 3.x.

* Update sprockets to 3.6.0.

* Make `DiscourseSassCompiler` work with Sprockets 3.

* Use `Sass::Rails::SassImporterGlobbing` instead of haxxing our own.

* Moneky patch so that we don't add dependencies for our custom css.

* FIX: Missing class.

* Upgrade ember-handlebars-template.

* FIX: require path needs to share the same root as the folder's path.

* Bump discourse-qunit-rails.

* Update ember-template-compiler.js to 1.12.2.

* `prepend` is private in Ruby 2.0.0.
2016-04-18 10:47:52 +08:00
Robin Ward cc25716e47 FIX: Allow message format translations to be overridden 2016-04-08 14:49:50 -04:00
Guo Xiang Tan 90fde5053d FIX: Load Redis patch much earlier. 2016-03-11 17:29:00 +08:00
Guo Xiang Tan ced86aa890 FIX: N+1 query on topics view page. 2016-02-23 16:22:40 +08:00
Gerhard Schlager 8481548def FIX: Load fallback locales after reloading translations 2016-02-22 22:48:56 +01:00
Gerhard Schlager 80c9fa4dca FIX: Load translation overrides for more than one locale 2016-02-22 22:48:56 +01:00
Sam bbbb09a6fb FEATURE: start tracking information about migrations that run
This commit adds a new tracking table that lets us know

- When a migration ran
- What version Discourse was at
- How long it took
- What version Rails was at

The built in tracking in Rails is very limited, does not track this info
2016-02-17 18:08:25 +11:00
Guo Xiang Tan b8193e8e67 Remove Arel patch that has been merged upstream. 2016-01-22 12:24:36 +08:00
Robin Ward d1ebb9d0b5 FIX: I18n Fallbacks were not applying correctly 2015-12-23 12:09:18 -05:00
Sam b8a47a1129 FIX: if a string was missing catch the throw
if overrides were in play and we tried to translate a
missing string we would throw an exception
2015-12-09 17:10:51 +11:00
Robin Ward de88be2fbc Support for "Only show overridden" in site text customization 2015-11-30 15:25:08 -05:00
Robin Ward 5e93140f85 FEATURE: Can override any translation via an admin interface 2015-11-27 11:35:19 -05:00
Robin Ward 434deb1bd3 SECURITY: Backported XSS fixes from Handlebars 2015-11-24 16:08:08 -05:00
Robin Ward 1506eba28d Support for overriding client side translation keys 2015-11-20 17:14:01 -05:00
Robin Ward e168c5fde3 PERF: Much more performant, multisite aware I18n overrides 2015-11-19 16:36:59 -05:00
Robin Ward 7f4ca91e06 FIX: I18n overrides should be multisite aware 2015-11-18 16:05:53 -05:00
Robin Ward 3720783c1b Refactor to our own Discourse I18n backend
This removes some monkey patches and makes testing easier.
It will also support database backed I18n changes.
2015-11-13 16:35:02 -05:00
Sam 805120fc95 FEATURE: add connnection reaping based on maximum age
This feature ensures connections to the db are always attempted to be closed
after 600 seconds of idle time.
2015-10-17 11:29:16 +11:00
Gerhard Schlager 454a628d58 Add tests for I18n.exists? freedom patch 2015-09-23 20:41:51 +02:00
Gerhard Schlager 825be30844 FIX: I18n.exists? should use locale fallbacks 2015-09-19 23:33:11 +02:00
Régis Hanol 1b44924cb0 replace 'open_uri_redirections' gem with a single freedom_patches file 2015-08-18 14:56:36 +02:00
Sam 1f044350f6 stop running exec_sql through active record
this avoids double logging
2015-08-14 10:01:38 +10:00
Kane York ecfa17b5a7 FEATURE: Localization fallbacks (server-side)
The FallbackLocaleList object tells I18n::Backend::Fallbacks what order the
languages should be attempted in. Because of the translate_accelerator patch,
the SiteSetting.default_locale is *not* guaranteed to be fully loaded after the
server starts, so a call to ensure_loaded! is added after the locale is set for
the current user.

The declarations of config.i18n.fallbacks = true in the environment files were
actually garbage, because the I18n.default_locale was
SiteSetting.default_locale, so there was nothing to fall back to. *derp*
2015-07-15 10:17:36 -07:00
riking 428ff097f1 Add some documentation to fast_pluck.rb 2015-05-14 21:15:33 -07:00
Sam 51d82fc25d PERF: Active Record #pluck is twice faster
PERF: SqlBuilder bypasses AR converters and uses PG directly
2015-05-13 16:46:17 +10:00
Robin Ward d9e5302a16 FIX: HTMLBars compilation was broken in production mode 2015-05-11 11:20:46 -04:00
Arthur Neves 439d0d2e37
Check Rails.version instead of ENV
Like that we can have code that works on multiple Rails versions, and we
dont need to mix a new method on Kernel.
Also, this makes easier to have multiple versions.
For instance, before master was 4.2, which is not the case anymore, so
on the code we should check versions and not Environment variables
2015-04-28 22:27:47 -04:00
Robin Ward 5b3f99aa50 Don't blow up if Redis switches to READONLY 2015-04-24 14:37:16 -04:00
Sam 9191fbe9fb Upgrade ruby racer so we can remove our freedom patch 2015-04-15 09:21:52 +10:00
Sam d390603274 PERF: Fix memory leak
We used to leak some memory every time you cook a post

see: https://github.com/cowboyd/therubyracer/pull/336
2015-04-01 13:30:55 +11:00
Sam 830ce05fe6 PERF: simplify and shrink the translation cache
FIX: leaking objects into the translation cache causing sidekiq to grow
2015-03-30 16:31:36 +11:00
Sam 3c6aede1aa using wrong var here 2015-02-12 14:40:07 +11:00
Sam a5f46c98c1 PERF: only load locale files we are using
We used to load up all translations in all languages
2015-02-12 12:08:14 +11:00
Sam 8be2cb061e barber upgrade, get assets precompile working on 1.9 2014-12-17 09:31:41 -05:00