2134ca9031
This commit optimises the database query generated by
`TopicsFilter#filter_categories` when the `-category:*` filter is used.
Previously, the method will add the `topics.category_id NOT IN
(<category ids to be excluded>)` filter to the resulting query. However,
we noticed that the performance of the query degrades as the number of
rows in the `topics` table grow and when the number of category ids to be
excluded is large.
Sample of query we ran on a large database in production to demonstrate
the improvement:
Before:
```
SELECT topics.id FROM topics WHERE topics.category_id NOT IN (83, 136, 149, 143, 153, 165, 161, 123, 155, 163, 144, 134, 69, 135, 158, 141, 151, 160, 131, 133, 89, 104, 150, 147, 132, 145, 108, 146, 122, 100, 128, 154, 95, 102, 140, 139, 88, 91, 87) ORDER BY topics.id DESC LIMIT 5;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=27795.34..27795.34 rows=1 width=4) (actual time=29.317..30.165 rows=5 loops=1)
-> Sort (cost=27795.34..27795.34 rows=1 width=4) (actual time=29.316..30.163 rows=5 loops=1)
Sort Key: id DESC
Sort Method: top-N heapsort Memory: 25kB
-> Gather (cost=1000.10..27795.33 rows=1 width=4) (actual time=0.187..26.132 rows=73478 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Seq Scan on topics (cost=0.10..26795.23 rows=1 width=4) (actual time=0.013..22.252 rows=24493 loops=3)
Filter: (category_id <> ALL ('{83,136,149,143,153,165,161,123,155,163,144,134,69,135,158,141,151,160,131,133,89,104,150,147,132,145,108,146,122,100,128,154,95,102,140,139,88,91,87}'::integer[]))
Rows Removed by Filter: 77276
Planning Time: 0.140 ms
Execution Time: 30.181 ms
```
After:
```
SELECT topics.id FROM topics WHERE NOT EXISTS (
SELECT 1
FROM unnest(array[83, 136, 149, 143, 153, 165, 161, 123, 155, 163, 144, 134, 69, 135, 158, 141, 151, 160, 131, 133, 89, 104, 150, 147, 132, 145, 108, 146, 122, 100, 128, 154, 95, 102, 140, 139, 88, 91, 87]) AS excluded_categories(category_id)
WHERE topics.category_id IS NULL OR excluded_categories.category_id = topics.category_id
) ORDER BY topics.id DESC LIMIT 5 ;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.42..13.52 rows=5 width=4) (actual time=0.028..0.110 rows=5 loops=1)
-> Nested Loop Anti Join (cost=0.42..179929.62 rows=68715 width=4) (actual time=0.027..0.109 rows=5 loops=1)
Join Filter: ((topics.category_id IS NULL) OR (excluded_categories.category_id = topics.category_id))
Rows Removed by Join Filter: 239
-> Index Scan Backward using forum_threads_pkey on topics (cost=0.42..108925.71 rows=305301 width=8) (actual time=0.012..0.062 rows=44 loops=1)
-> Function Scan on unnest excluded_categories (cost=0.00..0.39 rows=39 width=4) (actual time=0.000..0.001 rows=6 loops=44)
Planning Time: 0.126 ms
Execution Time: 0.124 ms
(8 rows)
```
|
||
|---|---|---|
| .devcontainer | ||
| .github | ||
| .vscode-sample | ||
| app | ||
| bin | ||
| config | ||
| db | ||
| docs | ||
| images | ||
| lib | ||
| log | ||
| migrations | ||
| patches | ||
| plugins | ||
| public | ||
| script | ||
| spec | ||
| test | ||
| vendor | ||
| .editorconfig | ||
| .eslintignore | ||
| .eslintrc.cjs | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .jsdoc | ||
| .licensed.yml | ||
| .licensee.json | ||
| .npmrc | ||
| .prettierignore | ||
| .prettierrc.cjs | ||
| .rspec | ||
| .rspec_parallel | ||
| .rubocop.yml | ||
| .ruby-gemset.sample | ||
| .ruby-version.sample | ||
| .streerc | ||
| .template-lintrc.cjs | ||
| Brewfile | ||
| CODEOWNERS | ||
| CONTRIBUTING.md | ||
| COPYRIGHT.md | ||
| Gemfile | ||
| Gemfile.lock | ||
| LICENSE.txt | ||
| README.md | ||
| Rakefile | ||
| config.ru | ||
| d | ||
| discourse.sublime-project | ||
| jsconfig.json | ||
| lefthook.yml | ||
| package.json | ||
| translator.yml | ||
| yarn.lock | ||
README.md
Discourse is the online home for your community. We offer a 100% open source community platform to those who want complete control over how and where their site is run.
Our platform has been battle-tested for over a decade and continues to evolve to meet users’ needs for a powerful community platform. Discourse allows you to create discussion topics and connect using real-time chat, as well as access an ever-growing number of official and community themes. In addition, we offer a wide variety of plugins for features ranging from chatbots powered by Discourse AI to functionalities like SQL analysis using the Data Explorer plugin.
To learn more, visit discourse.org and join our support community at meta.discourse.org.
Screenshots
Browse lots more notable Discourse instances.
Development
To get your environment set up, follow the community setup guide for your operating system.
- If you're on macOS, try the macOS development guide.
- If you're on Ubuntu, try the Ubuntu development guide.
- If you're on Windows, try the Windows 10 development guide.
- If you're looking to use a simpler Docker-based install, try the Docker development guide.
If you're familiar with how Rails works and are comfortable setting up your own environment, you can also try out the Discourse Advanced Developer Guide, which is aimed primarily at Ubuntu and macOS environments.
Before you get started, ensure you have the following minimum versions: Ruby 3.2+, PostgreSQL 13, Redis 7. If you're having trouble, please see our TROUBLESHOOTING GUIDE first!
Setting up Discourse
If you want to set up a Discourse forum for production use, see our Discourse Install Guide.
If you're looking for official hosting, see discourse.org/pricing.
Requirements
Discourse is built for the next 10 years of the Internet, so our requirements are high.
Discourse supports the latest, stable releases of all major browsers and platforms:
| Browsers | Tablets | Phones |
|---|---|---|
| Apple Safari | iPadOS | iOS |
| Google Chrome | Android | Android |
| Microsoft Edge | ||
| Mozilla Firefox |
Additionally, we aim to support Safari on iOS 15.7+.
Built With
- Ruby on Rails — Our back end API is a Rails app. It responds to requests RESTfully in JSON.
- Ember.js — Our front end is an Ember.js app that communicates with the Rails API.
- PostgreSQL — Our main data store is in Postgres.
- Redis — We use Redis as a cache and for transient data.
- BrowserStack — We use BrowserStack to test on real devices and browsers.
Plus lots of Ruby Gems, a complete list of which is at /main/Gemfile.
Contributing
Discourse is 100% free and open source. We encourage and support an active, healthy community that accepts contributions from the public – including you!
Before contributing to Discourse:
- Please read the complete mission statements on discourse.org. Yes we actually believe this stuff; you should too.
- Read and sign the Electronic Discourse Forums Contribution License Agreement.
- Dig into CONTRIBUTING.MD, which covers submitting bugs, requesting new features, preparing your code for a pull request, etc.
- Always strive to collaborate with mutual respect.
- Not sure what to work on? We've got some ideas.
We look forward to seeing your pull requests!
Security
We take security very seriously at Discourse; all our code is 100% open source and peer reviewed. Please read our security guide for an overview of security measures in Discourse, or if you wish to report a security issue.
The Discourse Team
The original Discourse code contributors can be found in AUTHORS.MD. For a complete list of the many individuals that contributed to the design and implementation of Discourse, please refer to the official Discourse blog and GitHub's list of contributors.
Copyright / License
Copyright 2014 - 2023 Civilized Discourse Construction Kit, Inc.
Licensed under the GNU General Public License Version 2.0 (or later); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:
https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Discourse logo and “Discourse Forum” ®, Civilized Discourse Construction Kit, Inc.
Accessibility
To guide our ongoing effort to build accessible software we follow the W3C’s Web Content Accessibility Guidelines (WCAG). If you'd like to report an accessibility issue that makes it difficult for you to use Discourse, email accessibility@discourse.org. For more information visit discourse.org/accessibility.
Dedication
Discourse is built with love, Internet style.