* Doing it in a post migration was a bad idea
because the migration will fail if the site
is down while trying to download uploads
which points to the instance. This mainly
affects self-hosters using `discourse_docker`
where `./launcher rebuild` will take the
existing container down.
This splits off the logic between SSO keys used incoming vs outgoing, it allows to far better restrict who is allowed to log in using a site.
This allows for better auditing of the SSO provider feature
* FEATURE: add branch option to remote theme import
* FIX: Add missing variable in params
* FIX: Add missing param for import_theme method
* SPEC: Add test methods for branch support in git import
* FIX: Add missing space to scss style
* Do not assume default branch as master
* Change branch field placeholder
* FIX: add missing div start tag
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.
The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.
```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```
To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.
- If force_lowercase_tags is disabled, then mixed case tags are allowed.
- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.
- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.
- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.
- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
Introduces a hidden setting (default is 0.1) that erodes bounce score
every time we send an email. This means that erratic failures are less
painful cause system auto corrects
Previously we used width and height for thumbnails, new code ensures
1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions
Also
- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
It was a dropdown to provide choices of color schemes,
and only one scheme could be shown.
With this commit, multiple color scheme previews can be displayed on
one page at the same time, making admins choose color schemes more
easily.
Theme preview windows are shrinked.
Imported default color schemes.
Co-Authored-By: Misaka 0x4e21 <misaka4e21@gmail.com>
```
change_column :topic_views, :ip_address, :inet, null: true
```
translates to
```
"ALTER TABLE \"topic_views\" ALTER COLUMN \"ip_address\" TYPE inet"
```
which locks the whole table while the migration is taking place.
### navigate_to_first_post_after_read setting for categories
When enabled on categories logged on users will return to OP after
reading the entire category. (useful for documentation categories)
### num_auto_bump_daily
Set a number of topics that will automatically bump daily on a category.
- Every 15 minutes we will check if any category has this setting
- Categories with the setting are shuffled
- We exclude pinned, closed, category description and archived topics
- Maximum of 1 topic for the list of categories is bumped till limit reached per category
- We always try to bump oldest first
- Limit is elastic using a RateLimiter that ensures that we only bump N per day
Also some minor organisation on category settings
Froze strings on category.rb
* Phase 0 for user-selectable theme components
- Drops `key` column from the `themes` table
- Drops `theme_key` column from the `user_options` table
- Adds `theme_ids` (array of ints default []) column to the `user_options` table and migrates data from `theme_key` to the new column.
- Removes the `default_theme_key` site setting and adds `default_theme_id` instead.
- Replaces `theme_key` cookie with a new one called `theme_ids`
- no longer need Theme.settings_for_client
Introduce new patterns for direct sql that are safe and fast.
MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API
- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder
See more at: https://github.com/discourse/mini_sql
* Feature: Push notifications for Android
Notification config for desktop and mobile are merged.
Desktop notifications stay as they are for desktop views.
If mobile mode, push notifications are enabled.
Added push notification subscriptions in their own table, rather than through
custom fields.
Notification banner prompts appear for both mobile and desktop when enabled.
Note, current practice if for plugins to submit PRs to core for any
migrations required for plugins, so we can better control schema.
Especially if core tables are being touched.
In this case index has close to zero cost unless akismet is installed
This reduces the akismet admin query from 20ms on every new page load to 0.5ms
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.
The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
This feature can be enabled by choosing a destination for the
`shared drafts category` site setting.
* Staff members can create shared drafts, choosing a destination
category for the topic when it is published.
* Shared Drafts can be viewed in their category, or above the
topic list for the destination category where it will end up.
* When the shared draft is ready, it can be published to the
appropriate category by clicking a button on the topic view.
* When published, Drafts change their timestamps to the current
time, and any edits to the original post are removed.
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.
New setting explicitly only removes from latest list but leaves the
category list alond
implemented review items.
Blocking previous codes - valid 2-factor auth tokens can only be authenticated once/30 seconds.
I played with updating the “last used” any time the token was attempted but that seemed to be overkill, and frustrating as to why a token would fail.
Translatable texts.
Move second factor logic to a helper class.
Move second factor specific controller endpoints to its own controller.
Move serialization logic for 2-factor details in admin user views.
Add a login ember component for de-duplication
Fix up code formatting
Change verbiage of google authenticator
add controller tests:
second factor controller tests
change email tests
change password tests
admin login tests
add qunit tests - password reset, preferences
fix: check for 2factor on change email controller
fix: email controller - only show second factor errors on attempt
fix: check against 'true' to enable second factor.
Add modal for explaining what 2fa with links to Google Authenticator/FreeOTP
add two factor to email signin link
rate limit if second factor token present
add rate limiter test for second factor attempts
Locking a post prevents it from being edited. This is useful if the user
has posted something which has been edited out, and the staff members don't
want them to be able to edit it back in again.
https://meta.discourse.org/t/search-logs-page/73281/11?u=techapj
This commit adds following features:
- support for tracking click through to user, tag and category
- new filter for search type (header, full page)
This commit also removes "most viewed topic" field from search logs page because we are now tracking multiple click through entities, so topic is not a special entity anymore. This also improves query perf. The query now takes `20.5ms` to runs, as opposed to `655.9ms` previously.
* Add user_home configuration option
* Use the new user_home preference to actually show the right home page
* Fix trailing whitespace
* Update user_option_serializer.rb
* Fix JavaScript default homepage tests
* Use an object instead of a giant switch
* Remove trailing whitespace
* Make the default `user_home` set to `null` instead of `0`
* Rename user_home to homepage_id
* Split alias levels in mentionable and messageable levels.
* Fixed some tests.
* Set messageable level to everyone by default.
* By defaults, groups are not mentionable or messageable.
* Made staff groups messageable by the system.
This change-set allows setting different defaults for different locales.
It also:
- Adds extensive testing around site setting validation
- raises deprecation error if site setting has the default property based on env
- relocated site settings for dev and tests in the initializer
- deprecated client_setting in the site setting's loading process
- ensure it raises when a enum site setting being set
- default_locale is promoted to `required` category.
- fixes incorrect default setting and validation
- fixes ensure type check for site settings
- creates a benchmark for site setting
- sets reasonable defaults for Chinese
There are 4 visibility levels
- public (default)
- members only
- staff
- owners
Note, admins and group owners ALWAYS have visibility to groups
Migration treated old "non public" as "members only"
Figuring out what unread topics a user has is a very expensive
operation over time.
Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)
When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.
This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.
We use pg functions, instead of nested queries here to assist the
planner.
This feature introduces the concept of themes. Themes are an evolution
of site customizations.
Themes introduce two very big conceptual changes:
- A theme may include other "child themes", children can include grand
children and so on.
- A theme may specify a color scheme
The change does away with the idea of "enabled" color schemes.
It also adds a bunch of big niceties like
- You can source a theme from a git repo
- History for themes is much improved
- You can only have a single enabled theme. Themes can be selected by
users, if you opt for it.
On a technical level this change comes with a whole bunch of goodies
- All CSS is now compiled using a custom pipeline that uses libsass
see /lib/stylesheet
- There is a single pipeline for css compilation (in the past we used
one for customizations and another one for the rest of the app
- The stylesheet pipeline is now divorced of sprockets, there is no
reliance on sprockets for CSS bundling
- CSS is generated with source maps everywhere (including themes) this
makes debugging much easier
- Our "live reloader" is smarter and avoid a flash of unstyled content
we run a file watcher in "puma" in dev so you no longer need to run
rake autospec to watch for CSS changes
- avoid access denied on bad cookie, instead just nuke it
- avoid marking a token unseen for first minute post rotation
- log path in user auth token logs
Revamped system for managing authentication tokens.
- Every user has 1 token per client (web browser)
- Tokens are rotated every 10 minutes
New system migrates the old tokens to "legacy" tokens,
so users still remain logged on.
Also introduces weekly job to expire old auth tokens.
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
previously we supported blanket read and write for user API, this
change amends it so we can define more limited scopes. A scope only
covers a few routes. You can not grant access to part of the site and
leave a large amount of the information hidden to API consumer.
FIX: warning about popup dimensions when using facebook login
Rules are:
- On account creation we always import
- If you already have an avatar uploaded, nothing is changed
- If you have no avatar uploaded, we upload from facebook on login
- If you have no avatar uploaded, we select facebook unless gravatar already selected
This also fixes SSO issues where on account creation accounts had missing avatar uploads
This feature ensures session cookie lifespan is extended
when user is online.
Also decreases session timeout from 90 to 60 days.
Ensures all users (including logged on ones) get expiring sessions.
Users can no longer opt-in for "public" edit history
if site owner disables it.
This feature adds cost and complexity to post rendering since
user options need to be premeptively loaded for every user in the
stream. It is also confusing to explain to communities with private edit
history.
- All unsubscribes go to the exact same page
- You may unsubscribe from watching a category on that page
- You no longer need to be logged in to unsubscribe from a topic
- Simplified footer on emails
* Rearrange frontend to account for mailing list mode
* Allow update of user preference for mailing list frequency
* Add mailing list frequency estimate
* Simplify frequency estimate; disable activity summary for mailing list mode
* Remove combined updates
* Add specs for enqueue mailing list mode job
* Write mailing list method for mailer
* Fix linting error
* Account for stale topics
* Add translations for default mailing list setting
* One query for mailing list topics
* Fix failing spec
* WIP
* Flesh out html template
* First pass at text-based mailing list summary
* Add user avatar
* Properly format posts for mailing list
* Move make_all_links_absolute into Email::Styles
* Apply first_seen_at to user
* Send mailing list email summary hourly based on first_seen_at
* Branch and test cleanup
* Use existing mailing list mode estimate
* Fix failing specs
We now optionally add a Variable Email Return Path to every email we send.
This allows us to cleanly handle email bounces, which in turn will improve
deliverability.
We no longer include previous replies as "context", instead
we include and excerpt of the post being replied to at the bottom
of notifications, this information was previously missing.
Users may opt in to emailing previous replies if they wish
or opt out of "in-reply-to" which makes sense in some email clients that
are smarter about displaying a tree of replies.
- always means we always send previous replies with every email
- never means we do not
- "unless previously sent" ... is the default, in which we only email you each reply once
The default_email_previous_replies site setting can control this toggle
This commit adds a new tracking table that lets us know
- When a migration ran
- What version Discourse was at
- How long it took
- What version Rails was at
The built in tracking in Rails is very limited, does not track this info
As it stands we load up user records quite frequently on the topic pages,
this in turn pulls all the columns for the users being selected, just to
discard them after they are loaded
New structure keeps all options in a discrete table, this is better organised
and allows us to easily add more column without worrying about bloating the
user table
UX: improve messaging so notifications list is far more stable
PERF: improve performance of notifcation lookup queries
- Add feature "SetTransientHeader" that allows shipping info to server
in the next Ajax request
- remove local storage hack used for notifications
- amend lookupStale to return hydrated objects, move logic into store
- stop magically clearing various notifications (likes, invitee accepted, group_summary, granted badge)
Messages are now in 3 buckets
- Inbox for all new messages
- Sent for all sent messages
- Archive for all messages you are done with
You can select messages from your Inbox or Sent and move them to your Archive,
you can move messages out of your Archive similarly
Similar concept applied to group messages, except that archiving and unarchiving
will apply to all group members
Re-organise user page so it is easier to find interesting info
split it into tabs
- Introduce notifications and messages tabs
- Stop couting stuff for the user page to speed up rendering
- Suppress more information when viewing your own profile
Since I18n has a DB backend now, I've introduced a helper we can use to
skip overrides in certain situations. Otherwise migration from empty
databases was broken.
Group owners are regular users that can add or remove users to a group
The Admin UX allows admins to appoint group owners
The public group UX will display group owners first and unlock UI to
add and remove members
Group owners can only be appointed on non automatic groups
Group owners may not appoint another group owner
- how long were people typing?
- how long was composer open?
- how many drafts were created?
- correct, draft saved to go away after you continue typing
store in Post.find(xyz).post_stat
- Increase size of email column to varchar(513)
- Give error message on signup when email is too large
Overall impact: Low, allows signups from blocked domains. Main risk is increased spam.
- add missing index to user actions for fast retrieval by post
- add missing indexes to users for fast retrieval of staff
- only refresh topic_users liked/bookmarked cache for affected users
FIX: keep the "uploading..." indicator until the server replies via the MessageBus
FIX: text was disapearing when uploading an avatar
PERF: always use a region for S3 (defaults to 'us-east-1')
FEATURE: ApplyCDN middleware when using S3
FIX: use the same pattern to store files on S3 and locally
PERF: keep a local cache of uploads when generating thumbnails
FEATURE: migrate_to_s3 rake task
FIX: history revision can now properly be hidden
FIX: PostRevision serializer is now entirely dynamic to properly handle
hidden revisions
FIX: default history modal to "side by side" view on mobile
FIX: properly hiden which revision has been hidden
UX: inline category/user/wiki/post_type changes with the revision
details
FEATURE: new '/posts/:post_id/revisions/latest' endpoint to retrieve
latest revision
UX: do not show the hide/show revision button on mobile (no room for
them)
UX: remove CSS transitions on the buttons in the history modal
FIX: PostRevisor now handles all the changes that might create new
revisions
FIX: PostRevision.ensure_consistency! was wrong due to off by 1
mistake...
refactored topic's callbacks for better readability
extracted 'PostRevisionGuardian'
Kyle E. Mitchell
Maja Komel
Régis Hanol
Guo Xiang Tan
Guo Xiang Tan
Gerhard Schlager
Bianca Nenciu
Robin Ward
Bianca Nenciu
Kyle Zhao
Erin Kosewic
David Taylor
Arpit Jalan
Sam
OsamaSayegh
Kris
Vinoth Kannan
Joffrey JAFFEUX
Leo McArdle
riking
Misaka 0x4e21
Jeff Wong
Neil Lalonde
OsamaSayegh
typeoneerror
Michael Howell
Erick Guan
Jakub Macina
Pat David
Claas Augner
Joe Buhlig
cpradio
James Kiesel
Thorben Egberts
Ubuntu
Jason W. May
David McClure