Commit Graph

2694 Commits

Author SHA1 Message Date
Neil Lalonde fe39cdc90a FEATURE: when a post is deleted because a moderator agreed with flags, send a message to the post author 2018-07-24 17:17:56 -04:00
Robin Ward 7058205f70 FIX: Broken specs 2018-07-24 12:00:34 -04:00
Robin Ward 236243f38a SECURITY: Consider `0.0.0.0` a private IP 2018-07-24 11:16:27 -04:00
Joffrey JAFFEUX 7a3c541077
UX: Preview multiple color schemes in wizard (#6151)
It was a dropdown to provide choices of color schemes,
and only one scheme could be shown.
With this commit, multiple color scheme previews can be displayed on
one page at the same time, making admins choose color schemes more
easily.

Theme preview windows are shrinked.

Imported default color schemes.

Co-Authored-By: Misaka 0x4e21 <misaka4e21@gmail.com>
2018-07-24 09:00:20 -04:00
Guo Xiang Tan fad9c2b971 PERF: Move `EmailLog#reply_key` into new `post_reply_keys` table. 2018-07-24 13:51:53 +08:00
Davide Porrovecchio dd9d815178 FIX: Add User Api Key headers to CORS
- add User-Api-Key and User-Api-Client-Id to Access-Control-Allow-Headers
- update test
2018-07-24 10:28:23 +10:00
David Taylor eda1462b3b
FEATURE: List, revoke and reconnect associated accounts. Phase 1 (#6099)
Listing connections is supported for all built-in auth providers. Revoke and reconnect is currently only implemented for Facebook.
2018-07-23 16:51:57 +01:00
David Taylor 2dc3a50dac FIX: Do not update `last seen` time for suspended users 2018-07-18 16:04:57 +01:00
Régis Hanol 6d6e026e3c FEATURE: selectable avatars 2018-07-18 12:57:43 +02:00
Sam 379384ae1e FIX: never block /srv/status which is used for health checks
This route is also very cheap so blocking it is not required

It is still rate limited and so on elsewhere
2018-07-18 12:37:01 +10:00
Guo Xiang Tan 3874d40910 Prepare to drop `EmailLog#topic_id`. 2018-07-18 10:22:24 +08:00
Leo McArdle 21ebb1cd54 FEATURE: Secondary emails support. 2018-07-16 11:09:49 +08:00
Arpit Jalan b1082924b9 FIX: do not validate topic deletions 2018-07-13 22:53:36 +05:30
Guo Xiang Tan c722b07057 FIX: `/t/:topic_id/last` route did not return any posts. 2018-07-13 14:26:10 +08:00
Kyle Zhao 2901691e87 FEATURE: per-category approval settings (#5778)
- disallow moving topics to a category that requires topic approval
2018-07-13 12:51:08 +10:00
Guo Xiang Tan 79ba418edd DEV: Don't join on a thread forever. 2018-07-12 15:46:07 +08:00
Guo Xiang Tan 258e9e35ca PERF: Make mega topics work without a stream.
There are tradeoffs that we took here. For the complete
story see
https://meta.discourse.org/t/performance-improvements-on-long-topics/30187/27?u=tgxworld.
2018-07-12 12:46:12 +08:00
OsamaSayegh decf1f27cf FEATURE: Groundwork for user-selectable theme components
* Phase 0 for user-selectable theme components

- Drops `key` column from the `themes` table
- Drops `theme_key` column from the `user_options` table
- Adds `theme_ids` (array of ints default []) column to the `user_options` table and migrates data from `theme_key` to the new column.
- Removes the `default_theme_key` site setting and adds `default_theme_id` instead.
- Replaces `theme_key` cookie with a new one called `theme_ids`
- no longer need Theme.settings_for_client
2018-07-12 14:18:21 +10:00
Guo Xiang Tan 4163f9e61e DEV: Better clean up for PostgreSQL failover test. 2018-07-10 09:53:25 +08:00
Guo Xiang Tan 96aca6d7e6
Remove legacy vote post action code. (#6009) 2018-07-09 16:54:18 +08:00
Andrew Schleifer dba22bbde2 rollback changes
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
2018-07-06 17:16:40 -05:00
Andrew Schleifer f8b90226cb fix test for absolute_base_url change 2018-07-06 17:08:18 -05:00
Andrew Schleifer 52e9f49ec1 fix s3 subfolders harder
specifically, include the folder in absolute_base_url
2018-07-06 16:28:40 -05:00
Neil Lalonde 211981ef23 add specs for min_trust_to_create_tag set to staff and admin 2018-07-05 11:39:32 -04:00
Patrick Gansterer 28dd7fb562 FEATURE: Create hidden posts for received spam emails (#6010)
* Add possibility to add hidden posts with PostCreator

* FEATURE: Create hidden posts for received spam emails

Spamchecker usually have 3 results: HAM, SPAM and PROBABLY_SPAM
SPAM gets usually directly rejected and needs no further handling.
HAM is good message and usually gets passed unmodified.
PROBABLY_SPAM gets an additional header to allow further processing.
This change addes processing capabilities for such headers and marks
new posts created as hidden when received via email.
2018-07-05 11:07:46 +02:00
Sam 0408e87e00 remove uneeded specs 2018-07-05 15:34:58 +10:00
Sam b54ba4c952 FIX: mentions broken after adding an <abbr> tag
A previous shortcut used was not allowing for <abbr and other tags starting with a

If <abbr> appeared anywhere in the text all mentions would fail to link
2018-07-05 09:27:11 +10:00
Régis Hanol 272646c1df FIX: only show the sequential replies warning for regular posts 2018-07-04 22:51:19 +02:00
Neil Lalonde 24882ce1a5 make rubocop happy 2018-07-04 09:42:31 -04:00
Neil Lalonde f134701c7b FIX: user topic and post counts can become negative when staff deletes posts in personal messages 2018-07-04 09:31:16 -04:00
Sam e72fd7ae4e FIX: move crawler blocking into anon cache
This refinement of previous fix moves the crawler blocking into
anonymous cache

This ensures we never poison the cache incorrectly when blocking crawlers
2018-07-04 11:14:43 +10:00
Neil Lalonde e8a6323bea remove crawler blocking until multisite support 2018-07-03 17:54:45 -04:00
Kasia Bułat b71cf6d422 FEATURE: Add search not operator for tags. 2018-07-03 15:57:34 +08:00
Jeff Wong d7f6d37a98 refactor: promotion spec 2018-07-02 16:23:45 -07:00
hellekin 25cfc98b67 Fix 'asscoiated' typo
I know that **Naming is CRITICAL** and that **Refactoring only NOT welcome**.

But since I spotted this (consistent) typo and the change does not affect any
functionality -- I checked the presence of "asscoiated" in the code base, I
guess the first rule trumps the second one.

It also gave me a false pretext to bypass my reluctance to use Google forms and
sign de CLA. Typos hurt the eye.
2018-06-29 11:10:05 +10:00
Sam db14e10943 SECURITY: category badges should HTML escape names 2018-06-28 18:15:07 +10:00
Maja Komel ec3e6a81a4 FEATURE: Second factor backup 2018-06-28 10:12:32 +02:00
Guo Xiang Tan cfa7898c2d Rename `TopicView#last_read_post_id` to `TopicView#filtered_post_id`. 2018-06-27 12:33:57 +08:00
Guo Xiang Tan cb69888758 PERF: Don't pluck all the columns just to retrieve a single value. 2018-06-27 11:41:35 +08:00
Arpit Jalan 6bcdc3ba4b FEATURE: allow author to delete posts irrespective of post_edit_time_limit 2018-06-26 21:43:06 +05:30
Guo Xiang Tan 49ffc1eb61 Revert "PERF: Send down gaps as the relevant posts load instead of front loading."
This reverts commit 4c3352528e.
2018-06-26 12:54:14 +08:00
Guo Xiang Tan 4c3352528e PERF: Send down gaps as the relevant posts load instead of front loading. 2018-06-26 12:49:06 +08:00
Guo Xiang Tan 0b6a2e9d1f Remove force summary mode for megatopics for now.
The logic is too hairy and we can't reliably determine
when to force summary mode. Work is underway to improve
perf for megatopics so this will not be required
eventually.
2018-06-26 12:49:06 +08:00
Jeff Wong 41f76a74f8 FEATURE: send message when a user reaches tl1 2018-06-22 13:20:00 -07:00
Guo Xiang Tan f69356e628 FIX: Users can't "show all posts" in forced summary topics. 2018-06-22 11:32:45 +08:00
Guo Xiang Tan 9a7a079f4d Force summary mode when user enters at the top of megalodoon topics. 2018-06-21 15:18:52 +08:00
Guo Xiang Tan f7d22bad90 FEATURE: Forced summary mode for megalodon topics.
This is mainly done for performance reasons and megalodon
topics are usually a byproduct of imports where site setting
limits are not respected.
2018-06-21 14:00:20 +08:00
Sam f66efc601d FIX: cubot android devices were detected as crawlers 2018-06-21 10:56:46 +10:00
Guo Xiang Tan ff5fc3cb08 Use a fixed limit for mega topic posts count. 2018-06-20 16:58:52 +08:00
Guo Xiang Tan 9c925a66ff PERF: Don't display days ago on timeline for megatopics.
Analysis using `pg_stat_statements` showed this query
to be eating up a significant portion of CPU.
2018-06-20 16:25:54 +08:00
Sam 44091f20c6 DEV: allow for method deprecation using Discourse.deprecate
New method deprecator will ensure one log message an hour happens
for all deprecated method calls per call site

Also removes unused monkey patches to ActiveRecord::Base
2018-06-20 17:53:49 +10:00
Sam cb824a6b33 DEV: remove all calls to SqlBuilder use DB.build instead
This is part of the migration to mini_sql, SqlBuilder.new is being
deprecated and replaced with DB.build
2018-06-20 17:53:49 +10:00
Guo Xiang Tan 806f0ca19d FIX: URL with params for svg images should not be light boxed. 2018-06-20 10:47:14 +08:00
Sam 94124ee2a6 skip erratic spec 2018-06-20 10:08:06 +10:00
Sam 4d984a5a63 extra diagnostics for thread issues 2018-06-20 09:19:16 +10:00
Arpit Jalan aedc61a3b4 FEATURE: allow large icon to be uploaded in wizard 2018-06-19 21:08:02 +05:30
Michael Brown ae5d255f83 FIX: Reference example.com instead of somesite.com in examples
* somesite.com actually exists...
* example.com should be used in examples and is harmless to visit
2018-06-19 10:37:24 -04:00
Joffrey JAFFEUX 24c27b5321
FEATURE: adds a add_report method accessible in plugin.rb 2018-06-19 15:00:11 +02:00
Sam 5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Guo Xiang Tan 630b4570ef Add specs for `RateLimiter::LimitExceeded#description`. 2018-06-19 07:48:03 +08:00
Guo Xiang Tan c18b86d9b2 UX: Don't add light box for SVG images. 2018-06-18 17:11:06 +08:00
Arpit Jalan c7ee70941e FEATURE: show category page options on wizard 'homepage' step 2018-06-15 19:11:41 +05:30
OsamaSayegh 2427c0a17c FIX: theme CSS should recompile when theme uploads change 2018-06-15 13:12:09 +10:00
Sam 87fabdc2f3 FIX: correct pool reaper
This removes a freedom patch and replaces with a custom reaper thread
it also captures an issue where reaper would fail when connections where
empty
2018-06-14 18:22:02 +10:00
Sam 71aa20bd30 FIX: pool drainer to use Rails 5.2 implementation
old implementation did not reap abandoned connections
2018-06-14 15:54:48 +10:00
Robin Ward fd54c92a52 FEATURE: New site setting, whitelisted_link_domains
If provided, users who normally couldn't post links (say, due to a
low trust level), can post links to those specific hosts.
2018-06-13 16:11:22 -04:00
Guo Xiang Tan 7c173265d5 FIX: Don't clear connections on the same process. 2018-06-12 13:06:25 +08:00
Guo Xiang Tan 646ed87aba Clear all connections once master recovers. 2018-06-12 12:13:59 +08:00
Guo Xiang Tan fd75e54793 Disconnect the pool during failover and fallback. 2018-06-12 11:09:19 +08:00
Jeff Wong 4599cc8435 FIX: PM participants listed inline 2018-06-11 18:14:25 -07:00
Guo Xiang Tan bfa0f71e2b FIX: `Discouse.keep_readonly_mode` incorrect extends expiry. 2018-06-12 00:21:29 +08:00
Guo Xiang Tan 204db00563 Skip postgres failover tests until we figure out why. 2018-06-11 15:51:16 +08:00
Guo Xiang Tan f9761c41a9 DEV: Stablize postgresql fallback adapter spec. 2018-06-11 13:58:04 +08:00
Guo Xiang Tan 91557063d8 Fix the build. 2018-06-11 13:39:52 +08:00
Guo Xiang Tan 5656e8f366 FIX: Can't boot Discourse with a read-only PG connection. 2018-06-11 12:29:23 +08:00
Neil Lalonde b8cf0788c6 FIX: broken mailto href's in emails 2018-06-08 13:11:58 -04:00
Guo Xiang Tan 8e0c1c8782 Re-enable skipped specs. 2018-06-08 10:04:06 +08:00
Arpit Jalan f9ab3848ed FEATURE: support disabling emails for non-staff users 2018-06-07 18:31:08 +05:30
Sam 945cb90e7e update specs 2018-06-07 20:55:42 +10:00
Sam f331d2603d DEV: improve design of site setting default provider
This refactors it so "Defaults provider" is only responsible for "defaults"

Locale handling and management of locale settings is moved back into
SiteSettingExtension

This eliminates complex state management using DistributedCache and makes
it way easier to test SiteSettingExtension
2018-06-07 14:33:41 +10:00
Sam 89ad2b5900 DEV: Rails 5.2 upgrade and global gem upgrade
This updates tests to use latest rails 5 practice
and updates ALL dependencies that could be updated

Performance testing shows that performance has not regressed
if anything it is marginally faster now.
2018-06-07 14:21:33 +10:00
Arpit Jalan 46fc57222f FEATURE: improve handling of site setting secrets 2018-06-04 21:31:34 +05:30
Blake Erickson 7750b30016 FIX: Allow a user to remove their title
Somewhere there was a regression and a user couldn't remove their own
title. If they selected '(none)' in the UI it would say it was saved,
but it would not actually be updated in the db.
2018-05-31 17:16:52 -06:00
Guo Xiang Tan 7fc8a36529 DEV: Take 2 Queue jobs in tests by default.
On my machine this cuts the time taken to run our test suite
from ~11mins to ~9mins.
2018-05-31 16:23:23 +08:00
Guo Xiang Tan 56e9ff6853 Revert "DEV: Queue jobs in tests by default."
Too risky for now

This reverts commit be28154d3b.
2018-05-31 15:34:46 +08:00
Sam 5086fdc76d FIX: add protection for scss removal during upgrade
In some cases plugins would remove scss files or change them, but CSS
was still calculated based off stale data in old instance cache
2018-05-31 17:02:48 +10:00
Guo Xiang Tan be28154d3b DEV: Queue jobs in tests by default. 2018-05-31 14:45:47 +08:00
Guo Xiang Tan f623740ffc DEV: Stablize DiscourseRedis tests. 2018-05-30 14:45:19 +08:00
Guo Xiang Tan 543b7cddfb FIX: Extra comma resulted in Github auth email result being an array.
https://meta.discourse.org/t/github-2fa-flow-broken/88674
2018-05-30 12:15:12 +08:00
Guo Xiang Tan 81b5d61fa7 FIX: `topic_destroyed` web hook couldn't find topic. 2018-05-28 17:38:02 +08:00
Sam e501936405 FIX: search server side error in rare condition 2018-05-28 15:28:18 +10:00
Sam 9c91c2509e improve spec stability 2018-05-25 15:16:40 +10:00
Sam 610bfec73e DEV: correct fragile spec 2018-05-25 14:29:11 +10:00
Sam 80adc1ee80 DEV: stabilize site setting spec
side effects could cause specs to fail in rare conditions
2018-05-25 12:16:00 +10:00
Sam d366f8d888 remove hack that destabliazed tese suite 2018-05-24 10:48:16 +10:00
Andrew Schleifer 4be0e31459 fix s3_cdn_url when the s3 bucket contains a folder 2018-05-23 15:51:02 -05:00
Ryan Mulligan fac4bf2f85 ignore emails that are from the reply by email addresses (#5843) 2018-05-23 10:04:45 +02:00
Guo Xiang Tan ad9e0d6bea
Merge pull request #5848 from OsamaSayegh/fix-social-login-groups
FIX: apply automatic group rules when using social login providers
2018-05-23 08:17:42 +08:00
OsamaSayegh f6d412465b FIX: apply automatic group rules when using social login providers 2018-05-23 02:26:07 +03:00
Sam 1ac1ee4287 FEATURE: allow registration of an array custom field 2018-05-22 16:48:39 +10:00
Sam bcfd9cf8b5 attempt to stabilize spec 2018-05-22 16:15:24 +10:00
Guo Xiang Tan f21a47eadd Improve specs to assert for the right record instead of just a count. 2018-05-17 08:47:24 +08:00
Régis Hanol a9ebde5111 FEATURE: new 'staged' users list for admins 2018-05-17 01:52:49 +02:00
Arpit Jalan abcb6af8f9 FIX: scrub secret setting values from logs 2018-05-15 09:19:26 +05:30
Régis Hanol 2cf6fb7359 FIX: always unstage users when they log in 2018-05-13 17:00:02 +02:00
Régis Hanol 86eb3528ec FEATURE: clearer error message when receiving a reply to an old notification 2018-05-09 18:51:01 +02:00
Régis Hanol 6b1ff0edd3 FIX: always update bounce score (instead of doing it once per day) 2018-05-09 16:40:52 +02:00
Robin Ward 8262fc5d15
Merge pull request #5807 from discourse/min-flags-by-topic
FEATURE: New site setting `min_flags_staff_visibility`
2018-05-08 09:17:29 -04:00
Sam 858a266031 FIX: exact matching should also match on title 2018-05-08 15:59:03 +10:00
Guo Xiang Tan 8cf0f51eb2 UX: Display site settings shortcut for `poll` and `discourse-nginx-performance-report`.
https://meta.discourse.org/t/improving-admin-plugins/84585/29?u=tgxworld
2018-05-08 10:34:32 +08:00
Robin Ward ac60a84329 FEATURE: New site setting `min_flags_staff_visibility`
When set higher than 1, flags won't show up for staff in the admin
section unless the minimum threshold of flags on a post is reached.
2018-05-07 16:05:13 -04:00
Régis Hanol a98aae3bcd FIX: topic search wasn't working for unlisted topics 2018-05-07 11:43:55 +02:00
Sam 3a06cb461e FEATURE: remove support for legacy auth tokens 2018-05-04 10:12:10 +10:00
Jeff Wong 62a8904729
Feature: Include participants at the bottom of PM emails (#5797)
* Feature: Include participants at the bottom of PM emails

... as undecorated links.

https://meta.discourse.org/t/email-notification-recipients-unclear-when-pm-is-sent-to-multiple-users/26934/13?u=featheredtoast

Fix: missing translation for PM mentions

* display membership count as `group (count)`
2018-05-03 15:50:06 -07:00
Joffrey JAFFEUX 980972182f
dashboard next: caching, mobile support and new charts 2018-05-03 15:41:41 +02:00
Sam a0cd54750c FIX: inline [code] not handled properly
The text

a
[code]test[/code]

Would eat up the `test` text cause translation from inline to block
for replace rule was not properly handled
2018-04-26 15:18:22 +10:00
Régis Hanol ddb092f397 FIX: update mail gem to fix UTF-8 parsing issue 2018-04-25 21:53:37 +02:00
Robin Ward a5172a37e0 Allow staff members to enable safe mode, even if disabled 2018-04-25 11:49:57 -04:00
Sam 035312d501 FIX: specify path for dosp cookie 2018-04-24 11:07:58 -04:00
Guo Xiang Tan c148500d51 FIX: Deadlock when topic with auto close topic timers exceeds `auto_close_topics_post_count`. 2018-04-23 13:34:24 +08:00
Sam ded84a4b58 PERF: improve performance once logged in rate limiter hits
If "logged in" is being forced anonymous on certain routes, trigger
the protection for any requests that spend 50ms queueing

This means that ...

1. You need to trip it by having 3 requests take longer than 1 second in 10 second interval
2. Once tripped, if your route is still spending 50m queueuing it will continue to be protected

This means that site will continue to function with almost no delays while it is scaling up to handle the new load
2018-04-23 11:55:25 +10:00
Neil Lalonde 70f2c5d3fd FEATURE: move staff tags setting to tag group settings 2018-04-20 15:34:23 -04:00
Guo Xiang Tan 98d880b67a Missed a spot in 45fe5dc793 2018-04-20 13:59:19 +08:00
Arpit Jalan 9a912b9b35 fix the build 2018-04-20 00:39:12 +05:30
Arpit Jalan 91bf10bd12 FIX: create upload record for exported csv files 2018-04-20 00:27:49 +05:30
Neil Lalonde 5b93d69939 FIX: error when non-staff user edits their topic after a hidden tag is added to it 2018-04-18 12:51:25 -04:00
Arpit Jalan c61ce66411 fix the build 2018-04-18 13:38:45 +05:30
Sam 59cd7894d9 FEATURE: if site is under extreme load show anon view
If a particular path is being hit extremely hard by logged on users,
revert to anonymous cached view.

This will only come into effect if 3 requests queue for longer than 2 seconds
on a *single* path.

This can happen if a URL is shared with the entire forum base and everyone
is logged on
2018-04-18 16:58:57 +10:00
Arpit Jalan 3566c6f02b FIX: strip emoji string from slug 2018-04-18 11:32:32 +05:30
Neil Lalonde b87fa6d749 FIX: blacklisted crawlers could get through by omitting the accept header 2018-04-17 12:39:30 -04:00
Régis Hanol 2585ada5ca FIX: don't allow spaces in 'reply_by_email_address' site setting 2018-04-17 17:08:12 +02:00
Sam 9980f18d86 FEATURE: track request queueing as early as possible 2018-04-17 18:06:17 +10:00
Guo Xiang Tan 828bfd9d27 Add specs for c74c933996. 2018-04-17 10:08:39 +08:00
Arpit Jalan 0183656631 FIX: verify filtered tags when checking for category minimum required tags 2018-04-14 23:20:43 +05:30
Régis Hanol a0a06492d8 FIX: make get_hostname more lenient to user input 2018-04-12 17:09:09 +02:00
Régis Hanol 3c8b43bb01 FIX: non-oneboxed links on separate lines should stay on separate lines 2018-04-11 21:33:45 +02:00
Arpit Jalan 48d43b33cc add client side validation for category minimum_required_tags 2018-04-11 07:17:52 +05:30
Arpit Jalan 9ca6ebe8fe FEATURE: enforce tagging on categories 2018-04-11 07:15:24 +05:30
Joffrey JAFFEUX 45f657336e
FEATURE: adds support for loading existing core asset in pretty text 2018-04-10 08:37:16 +02:00
Neil Lalonde f6cfff3cea UX: user preferences allows users to choose which title to use from their badges and groups 2018-04-06 14:34:36 -04:00
jose-hms b87205831b FEATURE: Staged user moderation (#5721) 2018-04-06 11:41:25 +02:00
Gerhard Schlager f2d00e5eff FEATURE: Use Message-ID for detecting email replies to group
Ignores the site setting "find_related_post_with_key" and always tries to honor the `In-Reply-To` and `References` header for emails sent to a group.

The senders email address must be included in the `To` or `CC` header of a previous email sent to the group and the `Message-ID` of that email must be included in the current email's `In-Reply-To` or `References` header.
2018-04-05 11:00:38 +02:00
Arpit Jalan 10759677db FIX: when uploading image newuser restrictions should not apply to staff 2018-04-05 09:51:03 +05:30
Robin Ward d690ae0281 FIX: Broken specs too. Quotes are fragile! 2018-04-02 14:23:10 -04:00
Guo Xiang Tan 142571bba0 Remove use of `rescue nil`.
* `rescue nil` is a really bad pattern to use in our code base.
  We should rescue errors that we expect the code to throw and
  not rescue everything because we're unsure of what errors the
  code would throw. This would reduce the amount of pain we face
  when debugging why something isn't working as expexted. I've
  been bitten countless of times by errors being swallowed as a
  result during debugging sessions.
2018-04-02 13:52:51 +08:00
Neil Lalonde 7311023a52
Merge pull request #5700 from discourse/crawl-block
FEATURE: control web crawlers access with white/blacklist
2018-03-27 15:06:03 -04:00
Neil Lalonde 4d12ff2e8a when writing cache, remove elements from the user agents list. also return a message and content type when blocking a crawler. 2018-03-27 13:44:14 -04:00
Gerhard Schlager fcd352e089 FIX: Try fixing unparsable email addresses
The mail gem returns `UnstructuredField` when it fails to parse email addresses, but the `Receiver` always expects an `AddressList`.
2018-03-27 18:28:54 +02:00
Sam 31dea5d5fc correct flaky spec 2018-03-27 17:57:19 +11:00
Gerhard Schlager b945a2dc39 Call `on_drop` only when tables/columns are dropped 2018-03-27 13:18:13 +11:00
Gerhard Schlager 4ad401bac5 Ignore delay when first migration was < 10min ago 2018-03-27 13:18:13 +11:00
Gerhard Schlager cd17f60952 Improve specs for accidental table/column drops and renames 2018-03-27 13:18:13 +11:00
Gerhard Schlager 19c5afc69d Protect against accidental table renames 2018-03-27 13:18:13 +11:00
Neil Lalonde f2c060bdf2 FEATURE: option for tags in a tag group to be visible only to staff 2018-03-26 17:05:09 -04:00
Robin Ward f03b6bd8c9 FIX: Update `last_version_at` when publishing 2018-03-26 16:06:20 -04:00
Robin Ward d4296f33ff FIX: Publishing should update the public_version too 2018-03-26 15:46:25 -04:00
Robin Ward 2b161a2391 FIX: Don't include shared drafts in global latest 2018-03-26 10:43:55 -04:00
Arpit Jalan b75b6de982 FIX: respect nofollow settings for onebox links 2018-03-26 18:21:16 +05:30
Neil Lalonde a84bb81ab5 only applies to get html requests 2018-03-22 17:57:44 -04:00
Neil Lalonde ced7e9a691 FEATURE: control which web crawlers can access using a whitelist or blacklist 2018-03-22 15:41:02 -04:00
Gerhard Schlager eebe1d8c56 Allow delayed dropping and renaming of tables 2018-03-21 12:05:12 +01:00
Sam 6a3c8fe69c FEATURE: protect against accidental column or table drops
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.

The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
2018-03-21 15:43:32 +11:00
Robin Ward b9abd7dc9e FEATURE: Shared Drafts
This feature can be enabled by choosing a destination for the
`shared drafts category` site setting.

* Staff members can create shared drafts, choosing a destination
category for the topic when it is published.

* Shared Drafts can be viewed in their category, or above the
topic list for the destination category where it will end up.

* When the shared draft is ready, it can be published to the
appropriate category by clicking a button on the topic view.

* When published, Drafts change their timestamps to the current
time, and any edits to the original post are removed.
2018-03-20 17:15:26 -04:00
Vinoth Kannan c5d26992d4 Prefer to use primary email for new user creation over other available emails 2018-03-19 17:10:35 +05:30
Guo Xiang Tan ec57ca54b5 FEATURE: Admins should be able to view PMs of any group. 2018-03-19 14:12:01 +08:00
Guo Xiang Tan 7fad30dacc Refactor test case. 2018-03-19 13:39:29 +08:00
Guo Xiang Tan aa91bd61a7 Improve specs for `TopicQuery#list_group_topics`. 2018-03-16 16:18:26 +08:00
Vinoth Kannan 58bb3967e5 SECURITY: Oneboxer should escape the URL before processing 2018-03-15 19:57:55 +05:30
Guo Xiang Tan a35227918f UX: Display group topics in a topic list. 2018-03-15 11:37:55 +08:00
Robin Ward 135195363b FIX: Not logging old post contents properly 2018-03-14 15:01:36 -04:00
Neil Lalonde 58508e553d FIX: tag input should not include tags you've already chosen in the search results 2018-03-13 17:17:16 -04:00
Robin Ward 31a0c4a9be FEATURE: Add `quote-modified` class if a quote has been modified 2018-03-13 13:41:06 -04:00
Robin Ward 65ac80b014 FEATURE: Log Staff edits in Staff Action Logs
Why? Some edits by staff are not tracked. For example, during the grace
period, or via the flags/silence dialog.

If a staff member is editing someone else's post, it now goes into the
Staff Action Logs so it can be audited by other staff members.
2018-03-12 13:51:40 -04:00
Gerhard Schlager d243b82fb3 FIX: Calculation of text length for <details> in excerpt was wrong 2018-03-12 16:55:23 +01:00
Sam 758b9a7dda FEATURE: prototype of local theme directory watcher
(note this will be documented a bit late)
2018-03-12 18:36:06 +11:00
Arpit Jalan f862122978 FIX: do not log personal message view if there exists a similar log in previous hour 2018-03-11 09:23:32 +05:30
Sam 5b6e49ae1d FEATURE: split out max diff to 2 settings
We trust staff + tl2 and up to perform edits in grace period.
Allow them significantly more edit room in grace period prior to storing
a revision.

editing_grace_period_max_diff_high_trust applies to users with tl2 and up.

So

tl0 / 1 : we store an extra revision if more than 100 chars change
tl2 and up : we store an extra revision if more than 400 chars change

We may tweak these numbers as we go.
2018-03-09 11:58:50 +11:00
Arpit Jalan a8149f8969 FIX: user should not be able to invite to PM if trust level requirment not met
FIX: when personal messages are disabled let user invite to a public topic
2018-03-08 14:59:04 +05:30
Sam e162cd16b6 FEATURE: editing_grace_period_max_diff to force revisions in grace period
If a user performs a substantive edit of 20 chars or more during grace period
we will store a revision to track the change

This allows for better auditing of changes that happen during the grace period
2018-03-07 18:34:34 +11:00
Sam f0d5f83424 FEATURE: limit assets less that non asset paths
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
Robin Ward 17a615165c FIX: Don't lock wiki posts when they're edited 2018-03-05 14:50:06 -05:00
Arpit Jalan 003b03d939 allow staff to delete user if posts are 5 or less irrespective of delete_user_max_post_age 2018-03-05 23:31:29 +05:30
Joffrey JAFFEUX ce1994beea FIX: do not treat :: as a valid emoji 2018-03-05 15:35:24 +01:00
OsamaSayegh 282f53f0cd FEATURE: Theme settings (2) (#5611)
Allows theme authors to specify custom theme settings for the theme. 

Centralizes the theme/site settings into a single construct
2018-03-04 19:04:23 -05:00
Robin Ward cd6c5fc5fb FIX: Disable "Make Personal Message" if they are disabled 2018-03-02 20:28:39 -05:00
Robin Ward 730201d423 New interface to upsert custom fields 2018-03-02 12:45:52 -05:00
Régis Hanol 6a78669ca3 FIX: 'reply by email addresses' site settings should allow email addresses without a 'reply_key' when 'find related post with key' is disabled 2018-03-02 17:53:18 +01:00
Sam d39d2b9352 FEATURE: whitelist data for themes 2018-03-02 14:52:09 +11:00
Guo Xiang Tan 939180efa8 FIX: Missing 2FA guards when sso is enabled or when local login is disabled. 2018-03-02 10:39:10 +08:00
Guo Xiang Tan 4f301905b6 Make rubocop happy. 2018-03-02 10:15:53 +08:00
Robin Ward b3883f5c32 FIX: Don't lock a post on edit unless the raw changes 2018-03-01 20:40:19 -05:00
Sam 75172024ca SECURITY: ensure users have permission when moving categories 2018-03-02 12:13:27 +11:00
Guo Xiang Tan 81ca3677f7 Add guard for `nil` in our `RateLimiter`. 2018-03-01 13:20:42 +08:00
Guo Xiang Tan 5d9f9c2614 FIX: `RateLimiter` max of zero or less should raise rate limit exceeded. 2018-03-01 13:14:46 +08:00
Guo Xiang Tan e7a7356986 Remove ancient votes code that is no longer used. 2018-02-28 14:37:22 +08:00
Guo Xiang Tan 902c5d11cf FIX: Don't allow other flag actions after `notify_moderator` has happened.
https://meta.discourse.org/t/receiving-sorry-an-error-has-occurred-during-flagging-step-of-discobot-tutorial/77233/5
2018-02-28 11:27:56 +08:00
Sam f295a18e94 FIX: stop double counting net calls in logs 2018-02-28 10:45:11 +11:00
Régis Hanol fd33090646 FEATURE: automatically elides gmail quotes 2018-02-26 23:54:02 +01:00
Régis Hanol 26d5ae61dd FIX: handle <pre> inside <blockquote> in html_to_markdown 2018-02-26 23:28:02 +01:00
Neil Lalonde 3313072957 Remove censored_pattern site setting, which is replaced by watched words 2018-02-26 16:29:27 -05:00
Régis Hanol 3be0294465 FIX: local post onebox was always pointing to 1st post 2018-02-26 16:05:35 +01:00
Régis Hanol 7d7f6faf40 FIX: properly render emojis in local oneboxes 2018-02-26 11:16:53 +01:00
Arpit Jalan b9a669ba32 FIX: do not log personal message view if user can't see the message 2018-02-25 22:39:25 +05:30
Régis Hanol 0559a4736a FIX: don't double request when downloading a file 2018-02-24 12:35:57 +01:00
Robin Ward 69af881f7f New site setting `trusted_users_can_edit_others`
The default is true to keep with previous discourse behavior. If
disabled, high trust level users cannot edit the topics or posts of
other users.
2018-02-22 20:39:24 -05:00
Guo Xiang Tan dd26bbe868
Merge pull request #5610 from discourse/pm-tags
FEATURE: Allow staffs to tag PMs
2018-02-23 07:07:41 +08:00
Vinoth Kannan 7cbda949f1 REFACTOR: New spec tests and code improvement 2018-02-22 20:27:02 +05:30
Régis Hanol 7a13e50aa6 fix build 2018-02-22 11:17:49 +01:00
Gerhard Schlager 97e19a7d02 Fix the build 2018-02-21 11:26:41 +01:00
Guo Xiang Tan 8964e75ad6
Merge pull request #5612 from discourse/featheredtoast-two-factor-login
Featheredtoast two factor login
2018-02-21 15:00:10 +08:00
Guo Xiang Tan 14f3594f9f Review Changes for f4f8a293e7. 2018-02-21 14:55:49 +08:00
Arpit Jalan 94fb8094c6 further optimize spec
thanks @tgxworld for the review.
2018-02-21 11:32:40 +05:30
Régis Hanol 0799831dbe FIX: use the avatar of the post rather than the topic in local oneboxes 2018-02-20 19:49:39 +01:00
Arpit Jalan a4bc54a686 FIX: strip zero width spaces from topic title 2018-02-21 00:12:39 +05:30
Arpit Jalan ed422285f0 optimize spec 2018-02-20 22:03:13 +05:30
Sam 86d12bd44b FEATURE: search within title using in:title
Also

- Significantly improved search ranking, title is treated most strongly
- Adds tag names to the index
- Run search re-indexer more aggressively
- Re-index topic and all posts on category change
2018-02-20 14:41:21 +11:00
Régis Hanol 60ec483caa FIX: include title in local onebox when linking to a different topic 2018-02-19 22:40:14 +01:00
Arpit Jalan c419c26f56 FEATURE: new site setting 'max_emojis_in_title' 2018-02-19 18:15:26 +05:30
Gerhard Schlager b6277e208b FIX: Cookies header didn't have the right format 2018-02-19 12:46:57 +01:00
Régis Hanol 61930e092a FIX: support incoming emails with just an attachment 2018-02-16 18:14:56 +01:00
Sam 94b2c70c0d PERF: remove oga gem
oga gem is automatically required by the aws gem
the oga gem retains about 1mb of memory, aws now uses nokogiri

This also removes the html normalize from the pretty text specs that was
a fair bit buggy as the polls test shows.
2018-02-15 14:36:40 +11:00
Robin Ward b4aa0b096e FIX: Couldn't like staff when `allow_flagging_staff` was set 2018-02-14 15:46:04 -05:00
Sam f028ffaf29 SECURITY: correct local onebox category checks
Also removes ugly "source_topic_id" from cooked posts

Patch was authored by @zogstrip

Signed-off-by: Sam <sam.saffron@gmail.com>
2018-02-14 10:40:46 +11:00
Erick Guan 03b3e57a44 FEATURE: login by a link from email
Co-authored-by: tgxworld <tgx@discourse.org>
2018-02-13 16:14:39 +08:00
Robin Ward 4dfe659189 Rename `allow staff flags` to `allow flagging staff` 2018-02-12 15:27:26 -05:00
Robin Ward 6287631745 FEATURE: New site setting, `allow staff flags`, false by default
For some large communities, it makes sense to disable flagging of
staff posts.
2018-02-12 14:56:21 -05:00
Robin Ward dedeb2deb8 FIX: Don't show the link button in the composer if linking is disabled 2018-02-08 12:56:10 -05:00
Robin Ward 1bab15c757 FEATURE: A site setting for a minimum TL to post links 2018-02-06 18:07:58 -05:00
Robin Ward b2b6dc68a6 FEATURE: a setting to customize the minimum TL to flag a post 2018-02-06 17:12:27 -05:00
Sam Saffron df8e43abdd use lazy & instead of try
unregister ip skipper in test
raise if called when a skipper is in play
2018-02-06 10:38:15 +11:00
Robin Ward eefd226611 Add extensibility point to `request_tracker` to skip IP addresses
This is useful if you want to run a per IP rate limiter but want to be
able to skip some IPs with custom logic.
2018-02-05 17:49:40 -05:00
Arpit Jalan 7e48c47d37 rename 'enable_private_email_messages' to 'enable_personal_email_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan ff0376a80b rename 'enable_private_messages' to 'enable_personal_messages' 2018-02-01 13:25:29 +05:30
Arpit Jalan 25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
Sam ee0d3f15c1 FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify
New site settings:

enable_markdown_linkify: which is default on, auto links https:// and http:// and mail://

markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-02-01 13:22:38 +11:00
Régis Hanol 7d2283167a UX: only crops images taller than 18:9 instead of 16:9 2018-01-31 22:31:16 +01:00
Régis Hanol b2f18fc98f FIX: system user edits should not generate notifications 2018-01-30 22:21:07 +01:00
Maja Komel 018cb7f36b add a custom user onebox (#5542)
* add custom user onebox

* add specs
2018-01-30 11:03:08 +01:00
Maja Komel 330912e1e5 FIX: allowed href scheme link can start with a + (#5537)
* allowed href scheme link can start with a +

* allow tel:// links only to start with +

* add missing semicolon

* add test
2018-01-30 11:02:23 +11:00
Sam f3502853fa correct spec regression 2018-01-30 08:54:37 +11:00
Sam f946db4afe FIX: inline oneboxer min title length of 2
also: cache mini onebox misses as well to cut down traffic
2018-01-30 08:40:04 +11:00
Arpit Jalan 8ab585e25f add more tests for 'log private message views' feature 2018-01-29 13:11:20 +05:30
Sam fa5880e04f PERF: ability to crawl for titles without extra HEAD req
Also, introduces a much more aggressive timeout for title crawling
and introduces gzip to body that is crawled
2018-01-29 15:40:12 +11:00
Arpit Jalan 1f6adbea5c FEATURE: log private message views 2018-01-29 08:08:08 +05:30
Robin Ward 44e2038b53 Setting to automatically lock posts when edited by staff 2018-01-26 14:01:30 -05:00
Robin Ward 6b04967e2f FEATURE: Staff members can lock posts
Locking a post prevents it from being edited. This is useful if the user
has posted something which has been edited out, and the staff members don't
want them to be able to edit it back in again.
2018-01-26 14:01:30 -05:00
Arpit Jalan 7b4e6d508b improve reviving_old_topic education message 2018-01-26 00:06:53 +05:30
Gerhard Schlager eb52c5469e FEATURE: Allow plugins to register a new locale 2018-01-25 14:57:41 +01:00
Gerhard Schlager ce060e2b86 FIX: Server didn't use default_locale as fallback locale 2018-01-25 14:57:41 +01:00
Sam adae963751 ensure we do not override charset for content type 2018-01-25 18:43:42 +11:00
Sam 3492a91056 FEATURE: allow site operators to disable emoji shortcuts 2018-01-24 12:21:44 +11:00
Robin Ward 782d75069e FIX: UX improvements for system messages when PMs are disabled 2018-01-23 13:12:11 -05:00
Robin Ward 17ebfd1715 FIX: Don't show suggested messages if private messages are disabled 2018-01-23 12:05:44 -05:00
Régis Hanol cbb321658f FIX: support for generating excerpt when nesting <details> blocks 2018-01-22 19:17:35 +01:00
Sam f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam fc36f095a7 FIX: ensure proper header transfer (except for cache control)
allows discourse special headers to be visible on hijacked reqs
2018-01-21 14:26:42 +11:00
Sam 12872d03be PERF: run post timings in background
This means that if a very large amount of registered users hit
a single topic we will handle it gracefully, even if db gets slow.
2018-01-19 08:27:29 +11:00
Gerhard Schlager 2a22b90538 SECURITY: email domain whitelist could be bypassed 2018-01-17 21:45:32 +01:00
Robin Ward 34ed6088b9 FEATURE: New modal to show flags received for a user 2018-01-17 15:08:08 -05:00
Arpit Jalan e04fb9a877 fix the build 2018-01-17 12:57:33 +05:30
Arpit Jalan 79eb9d7086 FEATURE: show header search results on search log term details page 2018-01-17 12:47:16 +05:30
Arpit Jalan 1208254961 FIX: validate presence of 'top menu' setting 2018-01-17 01:43:53 +05:30
Sam d7657d8e47 correct specs, ensure crawler layout only applies to html 2018-01-16 16:28:11 +11:00
Sam 7b562d2f46 FEATURE: much improved and simplified crawler detection
- phase one does it match 'trident|webkit|gecko|chrome|safari|msie|opera'
    yes- well it is possibly a browser

- phase two does it match 'rss|bot|spider|crawler|facebook|archive|wayback|ping|monitor'
    probably a crawler then

Based off: https://gist.github.com/SamSaffron/6cfad7ea3e6df321ffb7a84f93720a53
2018-01-16 15:41:45 +11:00
Sam 215c0d5569 FEATURE: allow system api to target users via external id or user id
usage ?api_key=XYZ&api_user_external_id=ABC
usage ?api_key=XYZ&api_user_id=123
2018-01-12 17:40:18 +11:00
Vinoth Kannan 988b13ac77 FIX: GitHub auth always asking to verify email for new users (#5487) 2018-01-12 15:17:29 +11:00
Gerhard Schlager 9f7ae908d8 Add specs to check email domain whitelist/blacklist for To and Cc 2018-01-10 16:57:26 +01:00
Sam cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Gerhard Schlager f086d28b30 FIX: Do not validate messages sent to mailing list mirror 2018-01-05 11:21:53 +01:00
Gerhard Schlager e0d73a957d FEATURE: Allow posting via email to read-only mailing list mirror category 2018-01-05 11:21:53 +01:00
Gerhard Schlager d7cd7e4dc7 FIX: Never mark emails sent to mailing list mirror as auto-generated 2018-01-05 11:21:53 +01:00
Gerhard Schlager ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Guo Xiang Tan 805d1c25d3
Merge pull request #5451 from tgxworld/treat_non_ascii_urls_as_valid
Treat non-ascii URLs in `UrlValidator`.
2017-12-27 14:14:20 +08:00
Sam a9e2fc59c4 FIX: [constructor] bbcode would cause markdown crash 2017-12-27 16:11:30 +11:00
Arpit Jalan ef4c6c67ba fix the build 2017-12-23 14:42:40 +05:30
Arpit Jalan 0514ac4ee2 FIX: verify presence of 'sso url' before enabling 'enable sso' 2017-12-23 13:30:49 +05:30
Régis Hanol d6b22e6cc1 FIX: whitelist oneboxed iframes 2017-12-23 01:56:33 +01:00
Guo Xiang Tan 4b51871f6a Treat non-ascii URLs in `UrlValidator`. 2017-12-21 14:22:55 +08:00
Guo Xiang Tan 6ecf37c482 Improve URL validation to check for a valid host.
Parsing a URL with `URI` is not sufficient as the following cases
are considered valid:

URI.parse("http://https://google.com")
=> #<URI::HTTP http://https//google.com>
2017-12-21 13:50:15 +08:00
Robin Ward 21e1b05c7e FIX: Don't disable details when below truncate limit 2017-12-20 15:45:00 -05:00
Robin Ward a0aca83c12 FIX: Broken spec 2017-12-19 17:55:41 -05:00
Robin Ward b3fda0ea86 FIX: details tags broke excerpts 2017-12-19 17:28:55 -05:00
Sam 57a1190b07 FIX: correct issue with search omitting words with multiple dots
Previously we used to break up words with dots incorrectly leading to
missing search terms
2017-12-19 16:04:24 +11:00
Sam 81b3a4a3da improve spec 2017-12-15 11:42:51 +11:00
Guo Xiang Tan f2565f6c7e SECURITY: Any group can be invited into a PM. 2017-12-14 14:57:48 +08:00
Sam 67aecff59c FEATURE: store twitter supplied email for auditing 2017-12-14 15:54:32 +11:00
Gerhard Schlager e30851e45a Move escape_uri method to a more suitable place 2017-12-12 20:17:46 +01:00
Guo Xiang Tan 6ade508f39 FIX: Prevent 'rack.input' missing error. 2017-12-12 16:40:35 +08:00
Arpit Jalan ff6dda85b7 FIX: replace curly quotes to regular quotes in search terms 2017-12-12 11:17:28 +05:30
Sam 4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam 68d3c2c74f FEATURE: add global rate limiter for admin api 60 per minute
Also move configuration of admin and user api rate limiting into global
settings. This is not intended to be configurable per site
2017-12-11 11:07:22 +11:00
Sam 90a55d6f7c FIX: handle CORS in hijacked requests 2017-12-07 10:31:04 +11:00
Gerhard Schlager 16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
Kyle Zhao 5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Sam 995bf3c84e correct spec on Ruby 2.3 2017-12-05 07:04:41 +11:00
Sam 5a9622163d FIX: regression around rate limiter 2017-12-04 21:44:16 +11:00
Sam dd70ef3abf Revert "Revert "PERF: improve speed of rate limiter""
This reverts commit 2373d85239.
2017-12-04 21:23:11 +11:00
Sam 2373d85239 Revert "PERF: improve speed of rate limiter"
This reverts commit a9bcdd7f27.
2017-12-04 21:19:28 +11:00
Sam d041377ccf correct test that does not work with discobot 2017-12-04 18:20:05 +11:00
Sam a9bcdd7f27 PERF: improve speed of rate limiter
Also

- adds a global rate limiter option
- cleans up usage in tests
- fixes freeze_time so it handles clock_gettime
2017-12-04 18:17:30 +11:00
Guo Xiang Tan b18cc81609 Make rubocop happy. 2017-12-04 10:55:31 +08:00
Guo Xiang Tan 22140efa70 Tests are still leaking connection after skipping.
* Could be in the setup.
2017-12-04 10:46:30 +08:00
Guo Xiang Tan 4c8402c50f Skip test that is leaking connections. 2017-12-04 09:26:51 +08:00
Vinoth Kannan 7f2eeaf767 FIX: Password required flag should be cleared whenever clearing the raw password (#5384) 2017-12-01 15:19:24 +11:00
Guo Xiang Tan b1375ef44e Ensure that we disconnect connection in test. 2017-11-29 20:57:13 +08:00
Gerhard Schlager 44ee388070 FEATURE: omit images from og and twitter description tags 2017-11-28 21:34:02 +01:00
Jeff Wong b094894c94 Feature: Add service worker registration method to plugin API 2017-11-28 14:01:41 +08:00
Sam df84e1c358 Correctly track hijacked requests 2017-11-28 16:47:20 +11:00
Sam 0caa335ef0 FIX: Handle more cases where HTTP status is not correct
HTTP status was not correct with send_file which uses streaming
2017-11-28 11:00:13 +11:00
Sam ca7af7b88f FIX: displaying wrong avatar and letter avatar
correct regression where params and env is reused in production
2017-11-28 09:28:40 +11:00
Sam 608207b2e5 FEATURE: avatar proxy happens in background
This ensures that even if it is slow to download avatars site will
continue to work

Also simplifies hijack pattern
2017-11-27 17:43:24 +11:00
Guo Xiang Tan 2e04ef97d9 Fix the build. 2017-11-27 10:53:05 +08:00
Guo Xiang Tan a7030e80bf Skip randomly failing test. 2017-11-27 10:51:18 +08:00
Guo Xiang Tan 71942e4f62
Merge pull request #5352 from tgxworld/method_for_replica_postgres_connection
Expose `replica_postgresql_connection` to `ActiveRecord::Base`.
2017-11-27 08:38:10 +08:00
Sam e0e99d4bbd PERF: hijack onebox requests so they do not use up a unicorn worker 2017-11-24 15:31:40 +11:00
Guo Xiang Tan a509f466a0 Expose `replica_postgresql_connection` to `ActiveRecord::Base`. 2017-11-24 09:35:45 +08:00
Gerhard Schlager 613f4d737a FIX: updating topic stats failed silently for invalid topics 2017-11-23 18:47:45 +01:00
Sam 2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Guo Xiang Tan f7642e076d REFACTOR: Avoid duplicated logic on server and client. 2017-11-21 17:10:04 +08:00
Guo Xiang Tan 385372e384 Revert "PERF: Reduce number of topics to filter while querying for unread."
This reverts commit c06b782cab.
2017-11-20 11:49:09 +08:00
Gerhard Schlager 546b206da0 replace indentation with non-breaking spaces when cooking emails 2017-11-18 17:16:44 +01:00
Neil Lalonde 87ec11e298 FIX: more accurate counting of posts read. Skipping to the end of a topic does not count all posts as read in user stats. 2017-11-17 16:08:46 -05:00
Gerhard Schlager 1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager aea161fabd suppress rejection email when email was sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Régis Hanol 678e28794a FIX: properly handle too large & broken images in posts 2017-11-16 15:45:07 +01:00
Guo Xiang Tan 3ab6318e5f Disable postgresql adapter tests until we figure out what is leaking
connections.
2017-11-16 07:17:22 +08:00
Sam f52111f787 FEATURE: allow plugins to easily detect if running in Rack
Usage: Discourse.running_in_rack? to tell if rack was booted
2017-11-16 08:39:29 +11:00
Gerhard Schlager 9207dee69a FEATURE: escape HTML when cooking plaintext emails 2017-11-15 20:22:11 +01:00
Régis Hanol 648832a643 fix the build 2017-11-15 11:53:14 +01:00
Vinoth Kannan 7b494a65c9 NEW: large image placeholder added in cooked html (#5291) 2017-11-15 11:30:47 +01:00
Guo Xiang Tan 89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Sam 813e21d0e8 FIX: current user serializer consistently returns {} for custom_fields
Resolves: #5210
2017-11-15 11:55:37 +11:00
Régis Hanol 23baaa5dcc FIX: don't crop iPhone X screenshots 2017-11-14 20:37:27 +01:00
Robin Ward 971e302ff2 FEATURE: Support an end date for user silencing 2017-11-14 13:20:19 -05:00
Gerhard Schlager d3baae5365 removes whitespaces and uses scope 2017-11-13 15:23:24 +01:00
Sam 8a66446849 FEATURE: add overflow-y auto to Markdown tables 2017-11-13 17:52:15 +11:00
Sam 4f28c71b50 FIX: error setting tombstone bucket when set to old version 2017-11-13 15:36:45 +11:00
Sam 3ac7d041ae UX: generic onebox treats all square images as avatars and renders them smaller 2017-11-13 11:21:19 +11:00
Gerhard Schlager 5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager fc6de6863b WIP 2017-11-12 21:12:34 +01:00
Gerhard Schlager 4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Neil Lalonde 9dc9ca4ac0 FIX: be consistent with how first posts in topics are counted. do like DirectoryItem.refresh_period :all 2017-11-10 12:18:25 -05:00
Sam 06365023c4 FEATURE: new API to search for a user by email 2017-11-09 17:04:21 +11:00
Sam 2d5bf0705a PERF: exact email match bypass
instead of scanning full table when there is an exact email match ONLY
return the actual user.
2017-11-09 16:53:14 +11:00
Guo Xiang Tan 4bb454d889 FIX: JSON custom fields incorrectly being converted to an array.
https://meta.discourse.org/t/custom-fields-simultaneous-save-with-json-becomes-an-array/73647
2017-11-08 11:17:37 +08:00
Régis Hanol be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Arpit Jalan 8f560f0bf4
Merge pull request #5282 from vinothkannans/local_onebox_slash
FIX: Topic links onebox differently if end in /
2017-11-07 17:39:38 +05:30
Guo Xiang Tan d9602fe729 Fix incorrect severity field. 2017-11-07 11:24:54 +08:00
Guo Xiang Tan c9df21e131 FEATURE: Allow Unicorn logs to be JSON formatted. 2017-11-06 12:46:14 +08:00
Vinoth Kannan 230fec68ca FIX: Topic links onebox differently if end in / 2017-11-06 01:41:36 +05:30
Robin Ward 2f0c9793f1 FEATURE: Allow multiple html builders to be registered via plugins 2017-11-03 11:32:32 -04:00
ckeboss 93633865d9 Adds primary user group as a class to quote (#5285)
* Adds primary user group as a class to quote

This feature addition will add the class `group-PRIMARY_USER_GROUP` to
the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group
of the user being quoted. This is similar to the class that is added to
a `topic-post`.

* Remove trailing whitespace

* Fix avatar in test

* Address PR comments

* Fix trailing whitespace
2017-11-03 09:51:40 -04:00
Guo Xiang Tan af01860a3d Clear all active connections after PostgreSQL failover tests. 2017-11-03 09:21:10 +08:00
Neil Lalonde 21dd2ccd43 FIX: only count regular posts in user stats when deleting 2017-11-02 18:05:23 -04:00
Neil Lalonde 30689783db don't decrement post_count for a post in a deleted topic that has already been uncounted 2017-11-02 18:05:23 -04:00
Neil Lalonde f29290ad11 FIX: don't count whispers in user stats post_count 2017-11-02 18:05:23 -04:00
Neil Lalonde 24af9b7d97 FIX: when a topic is deleted, update the post count stats of all user who replied 2017-11-02 18:05:23 -04:00
Robin Ward 64cb8a3ce3 FIX: Normalizer wasn't working with attributes without values 2017-11-02 14:11:20 -04:00
Gerhard Schlager 880d154381 FIX: deleting staged user of rejected email shouldn't delete incoming email 2017-10-31 15:13:23 +01:00
Guo Xiang Tan 53cadbdfc5 Skip failing tests on Travis first. 2017-10-28 00:22:41 +08:00
Guo Xiang Tan b636e858bb Fix randomly failing spec on Travis. 2017-10-27 23:53:58 +08:00
Sam 70aed105a6 FIX: bypass all site setting work for shadowed method 2017-10-27 11:12:44 +11:00
Erick Guan 7c3123a2dd Downcase encoded slug by default and more specs 2017-10-26 16:50:29 +08:00
Guo Xiang Tan 1c7b4381cc Improve postgresql failover specs. 2017-10-26 15:54:50 +08:00
Guo Xiang Tan 94782152c6 Ensure we clean up state in PostgreSQL failover specs. 2017-10-26 09:22:51 +08:00
Sam 877b7be579 FIX: in readonly mode don't double count pages 2017-10-25 13:19:43 +11:00
Neil Lalonde fb5b9b6d06 FIX: don't offer the "Something Else" flag reason to TL0 users since they don't have permission to send private messages 2017-10-24 11:47:42 -04:00
Neil Lalonde 4452d67a23 Revert "FIX: TL0 users' messages to moderators were not being posted when flagging private messages" 2017-10-23 18:17:53 -04:00
Panayotis Matsinopoulos 821ed23799 Use the CategoryHashtag::SEPARATOR (#5258)
Use the CategoryHashtag::SEPARATOR like in the previous examples.
2017-10-23 14:59:08 +02:00
Guo Xiang Tan 54455fa40b Use MessageBus to get other processes to failover faster. 2017-10-23 17:58:24 +08:00
Guo Xiang Tan 509ecbe9fb Fix test not starting with the right state. 2017-10-23 12:20:39 +08:00
Arpit Jalan 804b4f32f8 better error message when API authentication fails 2017-10-20 20:05:34 +05:30
Guo Xiang Tan fe1e78ddf4 Make PostgreSQL failover work with distributed cache. 2017-10-20 17:15:29 +08:00
Gerhard Schlager 1481462cbf PERF: Move oneboxing from cook method "email" to postprocessing 2017-10-20 10:26:45 +02:00
Guo Xiang Tan 057571d173 Raise errors in PostgreSQLFallbackHandler threads. 2017-10-20 12:06:03 +08:00
Robin Ward 838568cbc3 Refactor flag types for more customization 2017-10-19 13:55:23 -04:00
Guo Xiang Tan 25c25ae423 FEATURE: Allow user to leave a PM. 2017-10-19 12:32:55 +08:00
Guo Xiang Tan 79de10b212 FEATURE: Allow users to disable new PMs.
https://meta.discourse.org/t/is-it-possible-to-disable-private-messaging-for-a-specific-user/46391
2017-10-19 12:32:55 +08:00
Guo Xiang Tan 814c7ab503 Skip randomly failing tests first. 2017-10-19 12:25:50 +08:00
Régis Hanol cbdfc85466 FIX: images aren't lightboxed anymore (partially reverts 646c6eb7cd) 2017-10-18 23:54:36 +02:00
Sam a4c539bade FEATURE: Allow registration of detailed request logger
Detailed request loggers can be used to gather rich timing info
from all requests (which in turn can be forwarded to monitoring solution)

Middleware::RequestTracker.detailed_request_logger(->|env, data| do
   # do stuff with env and data
end
2017-10-18 12:10:30 +11:00
Sam 1dd2b51059 remove redundent stubs 2017-10-18 12:10:30 +11:00
Guo Xiang Tan 1b5ee0ae72 FIX: Couldn't boot Discourse app with a readonly postgres. 2017-10-17 13:44:30 +08:00
Sam Saffron 8185b8cb06 FEATURE: cache https redirects per hostname
If a hostname does an https redirect we cache that so next
lookup does not incur it.

Also, only rate limit per ip once per final destination

Raise final destination protection to 1000 ip lookups an hour
2017-10-17 16:22:54 +11:00
Sam Saffron 646c6eb7cd FEATURE: add :before_post_process_cooked hook
Also reduce amount of image downloading
2017-10-17 16:22:54 +11:00
Neil Lalonde 2db66072d7 SECURITY: signup without verified email using Google auth 2017-10-16 13:51:41 -04:00
Guo Xiang Tan 5f76e5062d Pause Sidekiq when postgres failovers. 2017-10-16 19:48:31 +08:00
Neil Lalonde b124e5f19f FIX: TL0 users' messages to moderators were not being posted when flagging private messages 2017-10-13 11:55:49 -04:00
Gerhard Schlager c0bb97b5cb FIX: delete staged users when the incoming email is rejected 2017-10-11 16:17:01 +02:00
Sam 4ea87b5ab8 Merge branch 's3_refactor' 2017-10-09 10:27:52 +11:00
Régis Hanol 3bdd8f57c1 FIX: invited staged users would sometimes not get notified of replies 2017-10-06 16:37:28 +02:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan b2127600fb Remove use of concurrent timer for Redis failover.
* Uses the same logic for Postgres failover.
2017-10-05 15:57:08 +08:00
Neil Lalonde f5a2ed99b0 FIX: deleting category background images sometimes has no effect 2017-10-04 17:04:37 -04:00
Neil Lalonde ddbd1d5ab8 allow regex options on username site settings 2017-10-04 15:08:51 -04:00
Neil Lalonde c29334cf23 FEATURE: the hide_email_address_taken setting works with the change email address form in user preferences 2017-10-04 11:41:25 -04:00
Sam 5881355006 remove uneeded assertion 2017-10-04 15:59:16 +11:00
Sam 4ee2fcd3d5 correct flaky spec 2017-10-04 10:47:24 +11:00
Gerhard Schlager 7f50380221 FIX: respect email domain whitelist/blacklist when creating staged users 2017-10-03 16:36:08 +02:00
Gerhard Schlager 76706f9144 FIX: don't create staged users when incoming email is rejected
FIX: don't send subscription mail to new users
2017-10-03 16:36:08 +02:00
Guo Xiang Tan 3e53dbcade UX: Only include tag hashtag postfix when necessary.
https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 13:54:50 +08:00
OsamaSayegh 79f3d299a1 Don't allow category definition topics to be converted to PMs (#5216) 2017-10-02 10:04:58 +02:00
Guo Xiang Tan b5bbb8ae8a Fix failing spec. 2017-10-02 13:16:01 +08:00
Guo Xiang Tan c872225762 Improve `MessageBus.track_publish` to allow filter by channel. 2017-10-02 11:34:57 +08:00
Sam f6fdc1ebe8 FEATURE: flexible crawler detection
You can use the crawler user agents site setting to amend what user agents
are considered crawlers based on a string match in the user agent

Also improves performance of crawler detection slightly
2017-09-29 12:31:50 +10:00
Guo Xiang Tan 8dae98a3f6 Skip randomly failing test on Travis for now. 2017-09-29 08:32:19 +08:00
Robin Ward 07e84a3afa FIX: Hack our title retriever so that it parses YouTube URLs 2017-09-28 09:30:22 -04:00
OsamaSayegh ab68379abb Use `PostDestroyer` to delete topics when performing bulk operation (#5202)
* Use `PostDestroyer` to delete topics when performing bulk operation

This PR should fix this bug: https://meta.discourse.org/t/bulk-deletion-of-topics-not-logged/50407?u=osama

* failing test
2017-09-28 14:22:49 +02:00
Sam 8ecf313a81 FIX: correctly raise errors when downloads fail
This corrects an issue where we are hitting Gravatar for 404 over and over

Also ensures file download properly reports errors
2017-09-28 16:35:43 +10:00
Guo Xiang Tan 426d2178c3 Fix undefined variable in `TopicCreator`. 2017-09-28 12:25:42 +08:00
Guo Xiang Tan d4388f54a2 FIX: Use exact patht to ensure we always redirect with the right format. 2017-09-28 10:29:41 +08:00
Guo Xiang Tan 5324c01209 FIX: Don't raise an error if reading from URL timeout. 2017-09-27 14:53:22 +08:00
Guo Xiang Tan 367fb1c524 FIX: Onebox fails on encoded URL.
https://meta.discourse.org/t/onebox-breaks-if-theres-chinese-text-in-url/67364
2017-09-26 18:34:54 +08:00
Guo Xiang Tan c25730935e FIX: Do not include non-human users on suspected users list.
https://meta.discourse.org/t/discobot-is-a-suspect-user/70558
2017-09-26 15:04:08 +08:00
Guo Xiang Tan c06b782cab PERF: Reduce number of topics to filter while querying for unread. 2017-09-26 12:36:52 +08:00
Robin Ward 561fa7d0cd FEATURE: Site Setting to hide suspension reason on the public profile 2017-09-25 12:25:14 -04:00
Robin Ward d7c37d9369 Add front end service for staff controls 2017-09-25 12:25:14 -04:00
Robin Ward 5e69217793 Add filtering support to flags 2017-09-25 12:25:14 -04:00
Guo Xiang Tan b8d9a6f747 Fix randomly failing spec. 2017-09-25 20:50:03 +08:00
Guo Xiang Tan 9540b952fd Fix broken condition in PostgresqlFallbackAdapter. 2017-09-25 13:48:59 +08:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan 1a9762a9c0 Fixed private group messages being exposed in suggested topics.
Revert "Revert "PERF: Avoid unnecessary expensive joins if possible.""

This reverts commit d9714c21c8.
2017-09-15 23:40:53 +08:00
Régis Hanol 0096ee40da FIX: only show participants the user can see 2017-09-13 17:14:03 +02:00
Gerhard Schlager 31ecb4fecf FIX: Handle incoming emails without email address in From header (#5177) 2017-09-12 22:35:24 +02:00
Guo Xiang Tan 07bfe3d053 FIX: Use a distrbuted cache for site locale. 2017-09-12 19:36:17 +08:00
Guo Xiang Tan 5d4221fbe1 PERF: Avoid calling expensive `PostGuardian#can_see_post?` multiple times.
Before

```
Your Results: (note for timings- percentile is first, duration is second
in millisecs)
---
topic_admin:
  50: 19
  75: 19
  90: 21
  99: 27
topic:
  50: 56
  75: 62
  90: 64
  99: 99
timings:
  load_rails: 1262
ruby-version: 2.4.1-p111
rss_kb: 198432
pss_kb: 136612
virtual: physical
architecture: amd64
operatingsystem: Ubuntu
memorysize: 15.59 GB
kernelversion: 4.10.0
physicalprocessorcount: 1
processor0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
rss_kb_9877: 327892
pss_kb_9877: 263671
rss_kb_9946: 325468
pss_kb_9946: 261671
rss_kb_10153: 326456
pss_kb_10153: 262657
```

After

```
Your Results: (note for timings- percentile is first, duration is second
in millisecs)
---
topic_admin:
  50: 18
  75: 18
  90: 20
  99: 28
topic:
  50: 41
  75: 42
  90: 46
  99: 49
timings:
  load_rails: 1201
ruby-version: 2.4.1-p111
rss_kb: 187936
pss_kb: 123596
virtual: physical
architecture: amd64
operatingsystem: Ubuntu
memorysize: 15.59 GB
kernelversion: 4.10.0
physicalprocessorcount: 1
processor0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
rss_kb_26478: 342360
pss_kb_26478: 276696
rss_kb_26547: 340368
pss_kb_26547: 275930
rss_kb_26747: 338964
pss_kb_26747: 274466
```
2017-09-08 14:07:24 +08:00
Guo Xiang Tan a0daa7cad0 Oops fix build again. 2017-09-07 18:59:02 +08:00
Robin Ward db929e58fc FIX: Don't allow staff to approve users with unverified emails 2017-09-04 12:55:39 -04:00
Sam Saffron e283e6aea0 FEATURE: allowed_iframes site setting for allowing iframes
This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:44 -04:00
Sam Saffron 3bdade8970 correct fragile spec 2017-08-31 15:55:56 -04:00
Sam c705159d22 Remove email column from user table 2017-08-29 11:50:56 -04:00
Guo Xiang Tan 37e5ee858f REFACTOR: Just pass a block instead of using a lambda. 2017-08-29 08:26:48 +08:00
Bianca Nenciu 6bc74ceb50 Split alias levels in mentionable and messageable levels. (#5065)
* Split alias levels in mentionable and messageable levels.

* Fixed some tests.

* Set messageable level to everyone by default.

* By defaults, groups are not mentionable or messageable.

* Made staff groups messageable by the system.
2017-08-28 12:32:08 -04:00
Bianca Nenciu bb3a5910d7 Support for sending PMs to email addresses (#4988)
* Added support for sending PMs to email addresses.

* Made changes after review.

* Added settings validator.

* Fixed tests.
2017-08-28 12:07:30 -04:00
Neil Lalonde 2c56f8df7c FEATURE: show tags in search results 2017-08-25 11:52:59 -04:00
Sam a4e3e081c9 correct spec 2017-08-24 10:40:14 -04:00
Guo Xiang Tan 68b8acbcfa Make Redis test less brittle. 2017-08-24 10:00:15 +09:00
Guo Xiang Tan d0f5a3d7a4 `Sidekiq::Testing.fake!` is the default mode. 2017-08-24 09:40:19 +09:00
Sam c7532003f3 add more diagnostics for flaky spec 2017-08-23 10:41:57 -04:00
Guo Xiang Tan 7c15b27a90 Merge pull request #5073 from xrav3nz/extract-github-avatar
FEATURE: import Github profile picture
2017-08-23 16:18:46 +09:00
Kyle Zhao 5868508e98 GH#retrieve_avatar: simplify conditional and restructured testing 2017-08-22 23:46:50 -04:00
Kyle Zhao 49f0119c12 FEATURE: import Github profile picture 2017-08-22 20:23:47 -04:00
Sam e8cdb1d8a7 correct flaky spec 2017-08-22 16:54:29 -04:00
Sam d4c28f6fcb remove stray puts from spec 2017-08-22 11:46:51 -04:00
Sam bcf7dc38c2 FEATURE: server side support for upload:// markdown
This allows uploads to be specified using short sha1 hash instead of full
URL

Client side change is pending
2017-08-22 11:46:23 -04:00
Guo Xiang Tan 2d4d76472d FIX: Broken suspect user lists.
https://meta.discourse.org/t/server-error-code-500/68404
2017-08-22 11:16:55 +09:00
Guo Xiang Tan e8f19e004e Use `find` instead. 2017-08-21 22:28:43 +09:00
Sam aeedecd27c correct erratically failing spec 2017-08-18 15:10:37 -04:00
Régis Hanol 629810bd07 FIX: when registering custom_field types, ensure we're casting them before checking for equality 2017-08-16 23:04:40 +02:00
Guo Xiang Tan 996ce8c443 Merge pull request #5024 from fantasticfears/site-setting
FIX: fails loud if default setting is not set
2017-08-16 09:36:38 +09:00
Erick Guan 1146772deb Fix: unlinked topic search model (#5044) 2017-08-15 11:46:57 -04:00
Erick Guan 1646bc0031 FIX: fails loud if default setting is not set
Noted:
- `push_api_secret_key` is set in initializer. Shimed with ''
- `default_theme_key` is set in seeding. Shimed with ''
2017-08-15 12:07:25 +02:00
Sam 6002f2ca4a FIX: not splitting words correctly for search tokenizer 2017-08-14 16:19:39 -04:00
Leo McArdle 0ef7a969f2 Some more HTML to Markdown fixes (#5046)
* FIX: handle spaces better within emphasis tags in html_to_markdown

* FIX: handle line breaks at beginning of emphasis tags in html_to_markdown
2017-08-14 22:13:24 +02:00
Régis Hanol 89e919ec42 UX: keep emojis in excerpts in flag queue 2017-08-14 15:16:47 +02:00
Rafael dos Santos Silva 5324c9817f FIX: Title prettify shoundn't downcase all non-ascii titles 2017-08-08 14:03:24 -03:00
Joffrey JAFFEUX 6cd8203686 FIX: allows onebox to force GET hosts returning wrong headers on HEAD 2017-08-08 11:44:27 +02:00
Guo Xiang Tan c5850422f0 FIX: SiteSettings defaults cache leaking across multisite. 2017-08-07 15:16:57 +09:00
Guo Xiang Tan 3f24ed2b3e Can't revert due to incompatibility of new site setting types.
Revert "Revert "FEATURE: Site settings defaults per locale""

This reverts commit 439fe8ba24.
2017-08-07 10:43:09 +09:00
Guo Xiang Tan 439fe8ba24 Revert "FEATURE: Site settings defaults per locale"
This reverts commit 468a8fcd20.
2017-08-07 10:31:50 +09:00
Leo McArdle a7b7fe335f FIX: allow an admin to click on blank errors (#5027)
* FIX: allow an admin to click on blank errors

* i18nlize strings

* what would a rails master do?
2017-08-04 20:04:26 +02:00
Leo McArdle 99527af38a FIX: show rejected emails with unrecognized errors (#5026)
Although 407a23663d will send rejection
messages for unrecognized errors, sometimes processing the email will
raise an error which has a blank message.

This commit:

1. Shows rejected emails which have already been processed and contain
   a blank error in /admin/email/rejected

2. Replaces new blank error messages with the error type
2017-08-04 16:20:44 +02:00
Guo Xiang Tan 8cc8010564 Maintain backwards compatibility before `Jobs::MigrateUploadExtensions` runs. 2017-08-03 11:56:55 +09:00
Guo Xiang Tan a47e297508 Merge pull request #5019 from tgxworld/more_resiliency_to_readonly_redis
Fix Redis command errors when trying to start app with a readonly Redis.
2017-08-03 06:34:36 +09:00
Sam 5942ad83c0 Skip inline onebox for domain only 2017-08-02 17:27:58 -04:00
Leo McArdle 65d5cd7239 FIX: generate valid markdown from <br></b> in an email (#5022)
* FIX: generate valid markdown from <br></b> in an email

* FIX: don't generate markdown for empty <strong> or <em> tags in emails
2017-08-02 23:02:59 +02:00
Sam a4e1920604 FIX: inline oneboxer not applying to lists 2017-08-02 16:10:08 -04:00
Sam f6bc572fb8 FEATURE: option to enable inline oneboxes for all domains
Also, change to prefer title over open graph which is often way too sparse
2017-08-02 14:27:31 -04:00
Erick Guan 468a8fcd20 FEATURE: Site settings defaults per locale
This change-set allows setting different defaults for different locales. 

It also:

- Adds extensive testing around site setting validation

- raises deprecation error if site setting has the default property based on env

- relocated site settings for dev and tests in the initializer

- deprecated client_setting in the site setting's loading process

- ensure it raises when a enum site setting being set

- default_locale is promoted to `required` category.

- fixes incorrect default setting and validation

- fixes ensure type check for site settings

- creates a benchmark for site setting

- sets reasonable defaults for Chinese
2017-08-02 12:24:19 -04:00
Guo Xiang Tan 9bc3038728 Fix Redis command errors when trying to start app with a readonly Redis. 2017-08-02 21:33:56 +09:00
Sam 97fa64d846 FIX: non tag/category # searches should be passed through 2017-08-01 18:15:14 -04:00
Sam 71ad3a48c2 Correct flaky distributed cache test
make distributed cache more testable
2017-08-01 16:11:48 -04:00
Guo Xiang Tan 5b76b9e888 Ensure that we cleanup after tests. 2017-08-01 23:42:26 +09:00
Guo Xiang Tan 7d5b6e2b3c Disable MessageBus's keepalive when Redis is readonly. 2017-08-01 23:07:52 +09:00
Neil Lalonde 24edbbb89c fix rubocop offenses 2017-07-31 20:34:02 -04:00
Neil Lalonde fa3c240e8b Merge pull request #4981 from dmacjam/fix_limited_search_results
FIX: limited search results
2017-07-31 20:23:57 -04:00
Régis Hanol 88ba052446 secure default for the 'find_related_post_with_key' site setting 2017-08-01 00:03:04 +02:00
Sam cdf7a020c1 correct mistake in spec 2017-07-31 16:40:59 -04:00
Neil Lalonde 7c1d7fb423 Merge branch 'master' into fix_limited_search_results 2017-07-31 15:55:31 -04:00
Erick Guan f1eccd113c Replace rmmseg gem for cppjieba_rb since better dictionary (#5006)
* Rename locale to ts config in search module to make it clear

* Replace rmmese-cpp for cppjieba_rb
2017-07-31 15:28:48 -04:00
Neil Lalonde 5d528f0d15 Merge pull request #4958 from dmacjam/search_posts_by_filetype
FEATURE: Search posts by filetype
2017-07-31 11:55:34 -04:00
Neil Lalonde 4a5907b116 Merge branch 'master' into search_posts_with_images 2017-07-31 10:44:41 -04:00
Jakub Macina f8edf2636c Fix rspec tests for search pagination. 2017-07-31 12:16:23 +02:00
Régis Hanol c7f76aa554 fix the receiver_spec 2017-07-31 11:29:39 +02:00
Arpit Jalan b059a0f789 extract url escaping to a dedicated class method and improved tests 2017-07-29 22:16:51 +05:30
Arpit Jalan 1fe553873c FIX: preserve fragment identifier when escaping url 2017-07-29 17:22:45 +05:30
Sam 56fd3e1cd4 attempt to stablise flaky spec 2017-07-28 16:59:37 -04:00
Robin Ward 9a76559345 FIX: Remove unused test 2017-07-28 11:59:12 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam 4f574e7c93 FEATURE: support inline bbcode blocks eg: [quote]test[/quote] 2017-07-27 17:55:17 -04:00
Neil Lalonde 68b3dd43ce fix intermittent failing tests, some watched word refactoring 2017-07-27 12:27:01 -04:00
Sam dc42d4411d correct flaky spec 2017-07-26 13:22:19 -04:00
Neil Lalonde 24cb950432 FEATURE: Watched Words: when posts contain words, do one of flag, require approval, censor, or block 2017-07-26 11:01:09 -04:00
David Taylor 27539e15e1 Allow whitespace around % scaler in markdown image syntax (#5000) 2017-07-26 10:54:20 -04:00
Neil Lalonde ea0e90b2b0 Merge branch 'master' into search_posts_with_images 2017-07-26 10:15:55 -04:00
Guo Xiang Tan d3a975e99a Improve specs to check top level stylesheets for SCSS compilation errors.
https://meta.discourse.org/t/fwd-growth-institute-pm-error-in-css-on-embed/66824/4
2017-07-26 10:50:51 +09:00
Guo Xiang Tan 96267f0845 Merge pull request #4960 from discourse/category-description-error
FIX: Explicit error when category description post is bad
2017-07-26 08:54:22 +09:00
Sam ad8643bfe2 attempt to stablise spec in travis 2017-07-25 16:58:19 -04:00
Sam ccbcbd0343 remove 2 failures that happen if discobot booted 2017-07-25 16:08:03 -04:00
Neil Lalonde d8c27e3871 Merge branch 'master' into search_posts_by_filetype 2017-07-25 14:41:20 -04:00
Sam f2d5bcd439 Stop writing to STDOUT/ERR during tests 2017-07-25 11:37:25 -04:00
Guo Xiang Tan 4cd9822611 Remove warnings in `ColumnDropper.mark_readonly` test. 2017-07-25 09:38:55 +09:00
Guo Xiang Tan be3c9bca0b Remove comment that is no longer relevant. 2017-07-25 08:58:40 +09:00
Sam 4da98cdcfe FIX: allow bbcode to be unquoted for default 2017-07-24 18:36:17 -04:00
Sam 04eac9f14a lets attempt to get these specs working! 2017-07-24 18:35:20 -04:00
Sam f67e715ef1 comment out specs that break others
will check in a fixed spec tomorrow
2017-07-24 17:28:24 -04:00
Sam c08a7aee8f clean up skipped tests
tighter connection handling in scheduler
2017-07-24 15:06:24 -04:00
Sam 80bb127cd3 use time zone for extra safety 2017-07-24 13:08:17 -04:00
Sam 7ebb9874a8 freeze time in spec 2017-07-24 13:00:15 -04:00
Sam 40abcfc2f5 FIX: smart quote edge case with quotes 2017-07-24 12:21:49 -04:00
Sam Saffron 045a2abcec FEATURE: remove the timecop gem
We should only have one way of mocking time, misuse of timecop
was causing build stability issues
2017-07-24 12:11:10 -04:00
Guo Xiang Tan d940166a89 Re-enable skipped `Scheduler::ScheduleInfo` test. 2017-07-25 00:03:03 +09:00
Guo Xiang Tan 8ee31d3c85 Re-enable skip tests for `Email::Styles`. 2017-07-24 23:16:54 +09:00
Guo Xiang Tan e82efc7b03 Fix `SiteSetingExtension` specs to coerce int to string. 2017-07-24 22:54:42 +09:00
Guo Xiang Tan 0ab0c82f11 Reenable skipped `Email::Receiver` spec. 2017-07-24 22:27:24 +09:00
Guo Xiang Tan d9c3ebe299 Reintroduce specs that wasn't causing random failures. 2017-07-24 19:32:38 +09:00
Régis Hanol 640ced004d FIX: wasn't unfreezing time in email processor spec 2017-07-24 11:01:53 +02:00
Guo Xiang Tan 1b0750d7ef Merge pull request #4983 from tgxworld/group_owners_can_invite_users_to_groups
Group owners can invite users to groups
2017-07-24 16:21:19 +09:00
Guo Xiang Tan fa09a02201 Fix specs. 2017-07-24 15:56:08 +09:00
Sam Saffron b9757bd02e remove readonly column specs for now 2017-07-22 22:20:57 -04:00
Robin Ward 2f8f2aa1dd FEATURE: Whitelists for inline oneboxing 2017-07-21 15:41:47 -04:00
Rafael dos Santos Silva 89ef5d36a9 FIX: Explicit error when category description post is bad 2017-07-21 16:07:29 -03:00
Régis Hanol fff5e2c3a5 Merge pull request #4984 from LeoMcA/unrecognized-error-email
FEATURE: send rejection email for unrecognized errors
2017-07-21 20:40:18 +02:00
Robin Ward 574681dc47 FIX: Show emoji in inline oneboxes 2017-07-21 14:24:48 -04:00
Leo McArdle 407a23663d FEATURE: send rejection email for unrecognized errors 2017-07-21 18:26:52 +01:00
Sam e1ce47a901 Pass the full CommonMark spec 2017-07-21 13:20:52 -04:00
Guo Xiang Tan 2a17f1ccd7 FIX: Group owners should be able to invite users to their groups.
https://meta.discourse.org/t/group-owner-cannot-send-an-invite-to-a-group/60617/12
2017-07-21 23:48:25 +09:00
Jakub Macina 4eb7f7cd10 Add rspec tests for search pagination. 2017-07-21 10:43:02 +02:00
Sam fc90dd3592 Work in progress, CommonMark spec 2017-07-20 17:47:16 -04:00
Robin Ward d2490cbbb8 Test failures for Inline Onebox 2017-07-20 16:01:16 -04:00
Robin Ward 3882722195 FEATURE: Inline (Mini) Oneboxing
see:
https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400?source_topic_id=66066
2017-07-20 15:38:04 -04:00
Sam 44fb2a2833 DEV: support multiple capture groups for text post process 2017-07-20 15:33:44 -04:00
Sam e7c170bb00 test v8 heap stats and aggregate all 2017-07-20 13:23:48 -04:00
Sam 1096dcd602 correct bbcode parsing edge case 2017-07-20 13:02:40 -04:00
Sam 911ffbb98c Correct mention edge cases
@sam. was not correctly detected as a mention
2017-07-20 11:52:38 -04:00
Leo McArdle d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Guo Xiang Tan b534778f46 FIX: Escape URL before attempting to resolve it. 2017-07-18 10:04:24 +09:00
Sam 627f5ddbd1 Correct specs 2017-07-17 18:00:25 -04:00
Sam c7b0764089 correct more edge cases with new engine 2017-07-17 17:44:48 -04:00
Sam 8921058c67 FIX: medium URL with @ incorrectly handled as mention
Also:

- remove unused code
- rename bbcode_ruler to bbcode.ruler
- add md.core.textPostProcess.ruler to apply at end of chain (excluding links)
2017-07-17 17:44:48 -04:00
Robin Ward 21e02d6969 Include the `search_log_id` in search results 2017-07-17 12:10:32 -04:00
Sam Saffron d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam 79a084dd58 Revert "remove old markdown engine work-in-progress"
This reverts commit ee470b5317.
2017-07-12 18:10:51 -04:00
Sam bcbb9f208d Revert "Integrate new engine, correct old specs"
This reverts commit f1b38ba4fb.
2017-07-12 18:10:07 -04:00
Sam f1b38ba4fb Integrate new engine, correct old specs
corrects edge cases with

- full quotes
- [url] with nested tags
- engine overrides
- onebox applying to non http srcs
2017-07-12 17:44:40 -04:00
Sam Saffron ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
Jakub Macina 677267ae78 Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests. 2017-07-12 17:19:27 +02:00
Sam 5d139e461c FIX: multi option poll not working (new engine) 2017-07-11 14:43:34 -04:00
Bianca Nenciu 1c917cc391 Added test. 2017-07-11 13:59:37 -04:00