A platform for community discussion. Free, open, simple.
Go to file
Alan Guo Xiang Tan 898b71da88
PERF: Add indexes to speed up notifications queries by user menu (#26048)
Why this change?

There are two problematic queries in question here when loading
notifications in various tabs in the user menu:

```
SELECT "notifications".*
FROM "notifications"
LEFT JOIN topics ON notifications.topic_id = topics.id
WHERE "notifications"."user_id" = 1338 AND (topics.id IS NULL OR topics.deleted_at IS NULL)
ORDER BY notifications.high_priority AND NOT notifications.read DESC,
  NOT notifications.read AND notifications.notification_type NOT IN (5,19,25) DESC,
  notifications.created_at DESC
LIMIT 30;
```

and

```
EXPLAIN ANALYZE SELECT "notifications".*
FROM "notifications"
LEFT JOIN topics ON notifications.topic_id = topics.id
WHERE "notifications"."user_id" = 1338
AND (topics.id IS NULL OR topics.deleted_at IS NULL)
AND "notifications"."notification_type" IN (5, 19, 25)
ORDER BY notifications.high_priority AND NOT notifications.read DESC, NOT notifications.read DESC, notifications.created_at DESC LIMIT 30;
```

For a particular user, the queries takes about 40ms and 26ms
respectively on one of our production instance where the user has 10K notifications while the site has 600K notifications in total.

What does this change do?

1. Adds the `index_notifications_user_menu_ordering` index to the `notifications` table which is
   indexed on `(user_id, (high_priority AND NOT read) DESC, (NOT read)
DESC, created_at DESC)`.

1. Adds a second index `index_notifications_user_menu_ordering_deprioritized_likes` to the `notifications`
   table which is indexed on `(user_id, (high_priority AND NOT read) DESC, (NOT read AND notification_type NOT IN (5,19,25)) DESC, created_at DESC)`. Note that we have to hardcode the like typed notifications type here as it is being used in an ordering clause.

With the two indexes above, both queries complete in roughly 0.2ms. While I acknowledge that there will be some overhead in insert,update or delete operations. I believe this trade-off is worth it since viewing notifications in the user menu is something that is at the core of using a Discourse forum so we should optimise this experience as much as possible.
2024-03-06 16:52:19 +08:00
.devcontainer
.github DEV: Update ember-cli to 5.6.0 (#25886) 2024-02-27 10:48:30 +01:00
.vscode-sample
app PERF: Add indexes to speed up notifications queries by user menu (#26048) 2024-03-06 16:52:19 +08:00
bin DEV: Drop Ember 3 feature flag 2024-02-26 12:22:05 +00:00
config DEV: Make `discourse_narrative_bot` use Rails autoload (#26044) 2024-03-06 11:14:53 +08:00
db PERF: Add indexes to speed up notifications queries by user menu (#26048) 2024-03-06 16:52:19 +08:00
docs FEATURE: Introduce APIs for manipulating header icons (#25916) 2024-03-04 12:51:49 -07:00
documentation
images
lib UX: Add sidebar icon (#26046) 2024-03-06 18:24:37 +10:00
log
migrations DEV: Extensively use `exception: true` in `system()` (#25911) 2024-02-27 11:33:28 +01:00
patches DEV: Update ember-cli to 5.6.0 (#25886) 2024-02-27 10:48:30 +01:00
plugins FIX: Update Discobot's `UserProfile#bio_raw` when default locale changes (#26045) 2024-03-06 11:42:20 +08:00
public Update translations (#25914) 2024-02-27 20:50:30 +01:00
script DEV: Improve generic import script (#25972) 2024-03-05 22:23:36 +01:00
spec PERF: Add indexes to speed up notifications queries by user menu (#26048) 2024-03-06 16:52:19 +08:00
test DEV: Update chrome-launcher from 0.15.2 to 1.1.0 (#25909) 2024-02-27 11:25:11 +01:00
vendor UX: Add sidebar icon (#26046) 2024-03-06 18:24:37 +10:00
.editorconfig
.eslintignore
.eslintrc.cjs DEV: Use `@discourse/lint-configs` (#24038) 2023-10-23 12:08:35 +02:00
.git-blame-ignore-revs
.gitattributes DEV: Remove GitHub gjs highlighting workaround (#24791) 2023-12-08 11:55:20 +00:00
.gitignore DEV: update .gitignore file (#25261) 2024-01-15 19:42:37 +05:30
.jsdoc DEV: Remove unmaintained tidy-jsdoc dependency (#25110) 2024-01-03 10:30:54 +00:00
.licensed.yml Build(deps): Bump highline from 2.1.0 to 3.0.0 (#25150) 2024-01-08 22:20:15 +01:00
.licensee.json DEV: Merge root JS packages (#25857) 2024-02-26 13:45:58 +00:00
.npmrc
.prettierignore DEV: Patch `deprecated-run-loop-and-computed-dot-access` in production (#25074) 2024-01-02 10:44:26 +00:00
.prettierrc.cjs DEV: Use `@discourse/lint-configs` (#24038) 2023-10-23 12:08:35 +02:00
.rspec
.rspec_parallel
.rubocop.yml DEV: Update rubocop-discourse to latest version 2024-03-04 15:08:35 +01:00
.ruby-gemset.sample
.ruby-version.sample
.streerc
.template-lintrc.cjs DEV: Update js linting setup (#25365) 2024-01-24 15:30:03 +01:00
Brewfile
CODEOWNERS DEV: Add "migrations-tooling" label to PRs for import scripts (#25062) 2023-12-28 21:26:05 +01:00
CONTRIBUTING.md
COPYRIGHT.md
Gemfile DEV: Update shoulda-matchers to 6.1.0 (#25362) 2024-01-22 12:12:51 +01:00
Gemfile.lock Build(deps-dev): Bump parallel_tests from 4.5.1 to 4.5.2 (#26036) 2024-03-05 22:43:56 +01:00
LICENSE.txt
README.md Replace Twitter screenshot with X screenshot (#25506) 2024-02-01 09:11:54 -07:00
Rakefile
config.ru
d
discourse.sublime-project
jsconfig.json Enable Embroider/Webpack code spliting for Wizard (#24919) 2023-12-20 13:15:06 +00:00
lefthook.yml
package.json DEV: Support `yarn dev` 2024-03-06 08:25:46 +08:00
translator.yml DEV: Fix translator-bot configuration for footnote plugin (#24065) 2023-10-23 23:10:17 +02:00
yarn.lock Build(deps-dev): Bump puppeteer-core from 22.3.0 to 22.4.0 (#26038) 2024-03-06 07:02:24 +08:00

README.md

Discourse is the online home for your community. We offer a 100% open source community platform to those who want complete control over how and where their site is run.

Our platform has been battle-tested for over a decade and continues to evolve to meet users needs for a powerful community platform. Discourse allows you to create discussion topics and connect using real-time chat, as well as access an ever-growing number of official and community themes. In addition, we offer a wide variety of plugins for features ranging from chatbots powered by Discourse AI to functionalities like SQL analysis using the Data Explorer plugin.

To learn more, visit discourse.org and join our support community at meta.discourse.org.

Screenshots

Discourse 3.1

Boing Boing

X Community

Mobile

Browse lots more notable Discourse instances.

Development

To get your environment set up, follow the community setup guide for your operating system.

  1. If you're on macOS, try the macOS development guide.
  2. If you're on Ubuntu, try the Ubuntu development guide.
  3. If you're on Windows, try the Windows 10 development guide.
  4. If you're looking to use a simpler Docker-based install, try the Docker development guide.

If you're familiar with how Rails works and are comfortable setting up your own environment, you can also try out the Discourse Advanced Developer Guide, which is aimed primarily at Ubuntu and macOS environments.

Before you get started, ensure you have the following minimum versions: Ruby 3.2+, PostgreSQL 13, Redis 7. If you're having trouble, please see our TROUBLESHOOTING GUIDE first!

Setting up Discourse

If you want to set up a Discourse forum for production use, see our Discourse Install Guide.

If you're looking for official hosting, see discourse.org/pricing.

Requirements

Discourse is built for the next 10 years of the Internet, so our requirements are high.

Discourse supports the latest, stable releases of all major browsers and platforms:

Browsers Tablets Phones
Apple Safari iPadOS iOS
Google Chrome Android Android
Microsoft Edge
Mozilla Firefox

Additionally, we aim to support Safari on iOS 15.7+.

Built With

  • Ruby on Rails — Our back end API is a Rails app. It responds to requests RESTfully in JSON.
  • Ember.js — Our front end is an Ember.js app that communicates with the Rails API.
  • PostgreSQL — Our main data store is in Postgres.
  • Redis — We use Redis as a cache and for transient data.
  • BrowserStack — We use BrowserStack to test on real devices and browsers.

Plus lots of Ruby Gems, a complete list of which is at /main/Gemfile.

Contributing

Build Status

Discourse is 100% free and open source. We encourage and support an active, healthy community that accepts contributions from the public including you!

Before contributing to Discourse:

  1. Please read the complete mission statements on discourse.org. Yes we actually believe this stuff; you should too.
  2. Read and sign the Electronic Discourse Forums Contribution License Agreement.
  3. Dig into CONTRIBUTING.MD, which covers submitting bugs, requesting new features, preparing your code for a pull request, etc.
  4. Always strive to collaborate with mutual respect.
  5. Not sure what to work on? We've got some ideas.

We look forward to seeing your pull requests!

Security

We take security very seriously at Discourse; all our code is 100% open source and peer reviewed. Please read our security guide for an overview of security measures in Discourse, or if you wish to report a security issue.

The Discourse Team

The original Discourse code contributors can be found in AUTHORS.MD. For a complete list of the many individuals that contributed to the design and implementation of Discourse, please refer to the official Discourse blog and GitHub's list of contributors.

Copyright 2014 - 2023 Civilized Discourse Construction Kit, Inc.

Licensed under the GNU General Public License Version 2.0 (or later); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:

https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Discourse logo and “Discourse Forum” ®, Civilized Discourse Construction Kit, Inc.

Accessibility

To guide our ongoing effort to build accessible software we follow the W3Cs Web Content Accessibility Guidelines (WCAG). If you'd like to report an accessibility issue that makes it difficult for you to use Discourse, email accessibility@discourse.org. For more information visit discourse.org/accessibility.

Dedication

Discourse is built with love, Internet style.